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Over a 4-1/2-year period, accreditors — independent agencies recognized by the 
Department of Education (Education) — sanctioned about 8 percent of schools for 
not meeting accreditor standards. They terminated accreditation for about 1 
percent of accredited schools, thereby ending the schools’ access to federal 
student aid funds. Accreditors must be officially recognized by Education as 
reliable authorities on assessing academic quality, and schools must be certified 
by these accreditors as meeting both academic and financial standards to qualify 
for federal student aid funds. From October 2009, when data were first 
consistently collected, through March 2014, when the latest data were released, 
GAO found that accreditors issued at least 984 sanctions to 621 schools, 
terminating the accreditation of 66 schools. Of sanctions issued in 2012, the most 
recent full year on reasons for sanctions at the time the study began, GAO found 
that accreditors most commonly cited financial rather than academic problems. 

A GAO analysis found that from October 2009 through March 2014, schools with 
weaker student outcomes were, on average, no more likely to have been 
sanctioned by accreditors than schools with stronger student outcomes. 
Researchers have reported that assessing multiple student outcomes could shed 
light on the quality of education provided by schools. Such outcomes are 
characteristics that Education and researchers consider important indicators of 
educational quality, but which accreditors are not necessarily required to use. On 
the other hand, accreditors were more likely to have sanctioned schools with 
weaker financial characteristics than those with stronger ones. With regard to 
academic quality, accreditors GAO interviewed reported that this area is difficult 
to oversee, saying that few quantifiable indicators exist. However, academic 
quality is a key accreditor responsibility under the Higher Education Act, and 
student aid funds may be at risk when schools that do not provide a quality 
education have access to these funds. The act prohibits Education from 
specifying the specific content of accreditor standards. However, Education must 
determine that these accreditor standards are effective in ensuring educational 
quality. GAO’s analysis raises questions about whether the standards accreditors 
use ensure that schools provide a quality education, and whether Education is 
effectively determining if these standards ensure educational quality. 

Education does not consistently use accreditor sanction information for oversight. 
For instance. Education does not systematically use sanction information to 
prioritize schools for in-depth review, as required by law. In addition, although 
Education’s 2014 strategic plan calls for better use of data in decision making, 
the department does not make consistent use of the accreditor sanction 
information it collects when it decides whether to re-recognize accreditors. While 
accreditor standards may be limited in enabling accreditors to effectively ensure 
academic quality, consistent use of accreditor sanction information could help 
Education determine whether schools are complying with federal financial aid 
requirements and oversee accreditors effectively. Moreover, federal internal 
control standards require agencies to assess risk and establish procedures to 
safeguard federal funds. 
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om 

441 G St. N.W. 
Washington, DC 20548 


U.S. GOVERNMENT ACCOUNTABILITY OFFICE 


December 22, 2014 

The Honorable George Miller 
Ranking Member 

Committee on Education and the Workforce 
House of Representatives 

Dear Mr. Miller: 

The U.S. Department of Education (Education) spent over $136 billion in 
fiscal year 2013 to help more than 14 million students pursue higher 
education. These funds were disbursed to students and their families 
through federal student aid programs authorized under Title IV of the 
Higher Education Act of 1965, as amended (Higher Education Act). To 
help ensure that schools with access to federal student aid provide a 
quality education, the act requires these schools to be accredited by 
entities recognized by Education as reliable authorities on assessing 
educational quality.^ These recognized accreditors must regularly assess 
the quality of schools they accredit, and take actions such as issuing 
sanctions when schools do not meet the standards that accreditors 
establish for them. Education also provides oversight of these federal 
funds by monitoring schools’ compliance with federal student aid 
requirements and collecting data on student outcomes and school 
finances.^ 

You requested that GAO examine issues related to accreditation. 

This study examines the following questions: 


^In order to participate in federal student aid programs authorized under Title IV of the 
Higher Education Act of 1965, as amended, schools must meet the act’s definition of an 
institution of higher education, which provides, in part, that in addition to being accredited, 
such institutions must be authorized to provide postsecondary education by the state in 
which they are located. Schools must also be determined to be eligible and certified by 
Education as meeting certain administrative capability and financial responsibility 
requirements. In this we report we are using the term “school” to refer to an institution of 
higher education. 

^In this report we refer to information collected by Education about a school’s students 
(e.g., graduation rates) as student outcome characteristics and to information collected 
about a school’s financial health (e.g., financial health scores) as financial characteristics. 
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1 . What sanctions have accreditors taken against schools that did not 
comply with accreditor standards? 

2. How likely are accreditors to sanction schools with weaker student 
outcome or financial characteristics by terminating their accreditation 
or placing them on probation? 

3. How does Education use information on accreditor sanctions to 
support its oversight efforts? 

To answer these questions, we reviewed relevant federal laws, 
regulations, and Education’s procedures for recognizing accreditors and 
overseeing schools. To determine what sanctions accreditors took 
against schools they accredit, we analyzed sanction data from 
Education’s accreditation database from October 2009 — when data were 
first consistently collected — through March 2014 — when the latest data 
were released at the time of our study, focusing on all the sanctions that 
accreditors reported to Education.^ We used data from Education’s 
Integrated Postsecondary Education Data System (IPEDS) to examine 
selected characteristics of sanctioned schools and the students these 
schools serve. To develop a better understanding of the reasons behind 
sanctions, we analyzed all termination and probation sanctions for fiscal 
year 2012, the most recent full year available when we began our study. 


^These data were collected in Education’s database of postsecondary schools that are 
accredited by Education-recognized agencies. The database was developed to help the 
public, including employers, make decisions about the quality of education provided by 
these schools. We included all terminations and probations in our analysis because all 
accreditors are required to report these sanctions to Education. We also included all show 
cause orders (which may indicate that a school must demonstrate why its accreditation 
should not be withdrawn) and warning sanctions (which may indicate a school does not 
meet one or more accreditation standards) reported to Education, which accreditors may, 
but are not required to, report to the department. In addition, we requested additional 
information on show cause orders and warning sanctions against member schools from 
accreditors that did not report these data to Education for fiscal years 2010, 201 1 , 2012, 
and 2013, and received responses from 27 of 31 accreditors. Based on these responses, 
we added an additional 90 show cause orders and 26 warning sanctions to our data set. It 
is difficult to know how many schools were placed on show cause or warning overall, 
because accreditors are not required to report these data to Education. 

^IPEDS is the federal government’s core postsecondary data collection effort. All 
postsecondary schools participating in federal student aid programs are required to 
complete a group of annual surveys on a variety of topics, ranging from total number of 
graduates to what percentage of revenue comes from financial aid. To ensure that we 
compared a consistent set of student outcomes and financial characteristics, we used 
data that were collected by Education for all schools that participate in federal student aid 
programs. 
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and a sample of other kinds of sanctions that accreditors reported to 
Education for that same timeframe.® 

To determine how likely accreditors are to sanction schools with weaker 
student outcome or financial characteristics, we conducted statistical 
modeling, using data Education collects on schools, to test whether 
schools with weaker characteristics — such as low graduation rates — are 
more likely to be terminated or placed on probation by accreditors. We 
analyzed characteristics data that Education collects in IPEDS and other 
Education data systems and that Education and researchers consider to 
be important indicators of educational quality. In addition, we examined 
federal internal control standards that call for agencies to establish and 
review ways to assess how they are evaluating their oversight, and to 
identify the risks they face in achieving their objectives, including risk from 
external parties.® 

To determine how Education used information on accreditor sanctions to 
support its oversight, we examined all schools with fiscal year 2012 
accreditor sanctions, the most recent full year of information on why 
accreditors issued sanctions that was available when we began our 
study, and drew upon data from Education’s Postsecondary Education 
Participants System (PEPS) database to determine if these sanctions 
prompted any follow-up monitoring or enforcement activity by Education, 
such as conducting in-depth reviews or imposing restrictions on a 
school’s access to federal student aid.^ For additional analysis, we 
selected a nonrandom sample of 10 schools that were sanctioned by their 
accreditors in fiscal year 2012 for reasons that could be related to federal 
student aid requirements. We reviewed Education’s information about the 


®We included all terminations and probations in our analysis because all accreditors are 
required to report these sanctions to Education. We also included a sample of 10 warning 
sanctions and 10 show cause orders. For these sanctions, we drew a nonrandomized 
sample from different accreditors to examine the reasons for these types of actions and 
determine if they differed from the reasons for terminations and probations. These 
reasons are not generalizable. It is difficult to know how many schools were placed on 
show cause or warning overall, because accreditors are not required to report these data 
to Education. 

®GAO, Standards for Internal Control In the Federal Government, GAO/AIMD-00-21.3.1 
(Washington, D.C.: November 1999). 

^This database contains Education’s oversight information of schools such as their 
eligibility for federal student aid programs, financial audit, and student loan default rate 
data. 
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cases, and discussed with Education officials what action they took or did 
not take as the result of these sanctions. While this sample is not 
generalizable, we believe it provides valuable examples about how 
Education has responded to information from accreditors. We evaluated 
Education’s efforts to oversee schools and accreditors using federal 
internal control standards for control activities, risk assessment, 
information, and communications. To assess the reliability of the 
accreditation database, the IPEDS data, and the PEPS data used in our 
analyses, we interviewed knowledgeable agency officials about the 
variables we used and conducted electronic testing to assess the 
reliability of these variables. We found these data sufficiently reliable for 
our purposes. 

We conducted this performance audit from July 2013 to December 2014 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 


Background 


Accreditors — generally nongovernmental, nonprofit organizations — play a 
critical role in protecting the federal investment in higher education as part 
of the “triad” that oversees schools participating in federal student aid 
programs authorized under Title IV of the Higher Education Act.® ® The 
roles and responsibilities of accreditors and the two other entities in the 
triad — Education and states — are intertwined under the Higher Education 
Act.''® However, each entity is typically understood to bear chief 
responsibility for a distinct aspect of the higher education oversight 
system, as seen in table 1. 


®Title IV of the Higher Education Act is codified at 20 U.S.C. §§ 1070 - 1099d and 42 
U.S.C. §§ 2751 -2756b. 

^Accrediting agencies are generally funded by dues and fees paid by member schools. 
^°20 U.S.C. §§ 1099a, 1099b, and 1099c. 
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Table 1: Oversight Roles and Responsibilities of the “Triad” 

Accrediting Agencies 

Apply and enforce standards that help ensure that the education offered by a postsecondary school is of 
sufficient quality to achieve the objectives for which it is offered. 

Education 

Recognize accreditors determined to be reliable authorities as to the quality of education offered by schools; 
certify schools as eligible to participate in federal student aid programs; and ensure that participating 
schools comply with the laws, regulations, and policies governing federal student aid. 

States 

Authorize schools to offer postsecondary education and respond to student complaints. 

Source: GAO analysis of relevant federal laws and regulations. | GAO-15-59 


Accrsditation ProC6SS The purpose of accreditation in the oversight triad is to help ensure that 

member schools meet quality standards established by accrediting 
agencies. While accreditation first arose in the U.S. as a means of 
ensuring academic quality by nongovernmental peer evaluation, today the 
process also serves as one of the bases for determining a school’s 
eligibility to participate in federal student aid programs.. Accreditation 
agencies and processes predate the Higher Education Act, and 
accreditation is a peer review process that serves several purposes in 
addition to being a gatekeeper for federal funds, including facilitating the 
transferability of courses and credits across member schools. According 
to representatives of schools and accrediting agencies, accreditation also 
encourages schools to maintain a focus on self-improvement. 

While Education is required to determine whether accrediting agencies 
have standards in certain areas before recognizing them, the accrediting 
agencies are responsible for evaluating member schools to determine if 
they meet the accreditors’ standards. This accreditation process generally 
occurs at least every 10 years, depending on the accreditor and the 
school. The process is typically conducted by volunteer peer evaluators, 
generally from other member schools, selected by the accreditor, with 
final accreditation decisions made by a board that includes 
representatives from member schools and the public. While specific steps 
vary by accrediting agency, schools generally go through a similar 
accreditation process (see fig. 1). 
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Figure 1 : Sample Accreditation Process 


D School self-evaluation 

School sends accrediting agency a 
self-evaluation describing its performance 
in relation to the accreditor's standards. 




Accreditation typicaiiy iasts from 
a few years to up to 10 years, 
after which the school must reapply 



Appeals 

process 

available 


B Peer review site visit 

A committee of peer volunteers, generally 
from other schools, is selected by the 
accrediting agency to review the 
school's self-evaluation and conduct 
a site visit to assess the school's 
compliance with accreditor's standards. 


Q Accrediting agency decision 

The accrediting agency considers the 
self-evaluation and the peer review team's 
assessment before issuing a decision on 
the school's accreditation status. 




E] Peer review report 

Committee produces a confidential report 
assessing the school's compliance with 
the accreditor's standards and makes 
a recommendation on the school's 
accreditation. 


Source: GAO analysis of information from accreditors and relevant federal law. | GAO-1 5-59 


Typ© of Accrsditors in general, two different types of accreditors — regional and national — offer 

accreditation to schools that allows the schools to access federal student 
aid funds. Regional accreditors accredit mostly nonprofit and public 
schools, while national accreditors generally accredit for-profit schools. At 
the time of our review, regional accreditors had 3,134 member schools in 


^^This study examines accreditors that are recognized to accredit entire schools, thereby 
allowing them to receive federal student aid funds, as opposed to accreditors that only 
accredit specific programs within schools. Education also recognizes some programmatic 
accreditors to accredit free-standing professional or vocational schools (such as law 
schools or schools of massage therapy) for federal student aid purposes. 

wide variety of postsecondary schools offering different types of postsecondary 
education programs may be eligible to participate in federal student aid programs. 
Postsecondary schools include: (1 ) public schools — operated and funded by state or local 
governments; (2) nonprofit schools — owned and operated by nonprofit organizations 
whose net earnings do not benefit any shareholder or individual; and (3) for-profit 
schools — privately owned, and net earnings can benefit shareholders or individuals. 
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total, while national accreditors had 3,719.''^ Seven regional accreditors 
accredit schools within a particular region and have historically accredited 
public and private nonprofit schools that award degrees. In addition, eight 
national accreditors operate nationwide and have historically accredited 
vocational or technical schools that do not award degrees. Differences 
between regional and national accreditors still exist, as seen in figure 2, 
but some for-profit schools have obtained regional accreditation in recent 
years and many for-profit schools currently award two- and four-year 
degrees. 


Figure 2: Membership of Regionai and Nationai Accreditors 


Regional 


National 



Public schools 
Nonprofit schools 


Source: GAO analysis of data from the Integrated Postsecondary Education Data System (IPEDS) and the Department of Education’s accreditation database. | GAO-1 5-59 


Note: Figure refers to accreditors that provide accreditation to schools that allows the schools to 
receive federal student aid funds, as opposed to accreditors that only accredit specific programs 
within schools. 


Accrsditor Standards The Higher Education Act lays out areas in which federally recognized 

accreditors must have standards for their members (see sidebar below). 


^^Programmatic and “other” accreditors, including state entities that can accredit schools, 
had a total of 1 ,334 member schools. For programmatic accreditors, 1 percent of their 
member schools were public, 66 percent were nonprofit schools, and 33 percent were for- 
profit schools. 
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and requires them to assess compliance of member schools with 
standards in these areas to ensure the quality of education offered.'''^ 


Areas in Which Accreditors Are Required to 
Have Standards 

1 . Success with respect to student achievement 
(Standards may be established by the school 
and differ according to its mission.) 

2. Curricula 

3. Faculty 

4. Facilities, equipment, and supplies 

5. Fiscal and administrative capacity 

6. Student support services 

7. Recruiting and admissions practices 

8. Measures of program length and objectives 

9. Student complaints 

10. Compliance with federal student aid program 
responsibilities 

Source: 20 U.S.C. § 1099b(a)(5) | GAO-15-59 


Education is required to assess whether an accreditor consistently 
applies and enforces standards in the specified areas that are effective in 
ensuring that the education offered is of sufficient quality to achieve its 
stated objective. However, the Higher Education Act does not dictate 
the specific content of the standards and prohibits Education from 
promulgating regulations with respect to such standards or establishing 
additional areas in which accreditors must judge schools’ compliance.^® It 
also prohibits Education from establishing criteria to specify, define, or 
prescribe standards accreditors use to assess any school’s success with 
respect to student achievement (otherwise known as student 
outcomes).''^ While accreditors are not required to use information on 
specific student outcome or financial characteristics of schools to assess 
whether schools are meeting their standards, and although no single 
outcome can be used to fully examine something as complex as 
educational quality, prior research and experts in the field of 
postsecondary education have reported that assessing multiple student 
outcomes (e.g., graduation rates and student loan default rates) can shed 
light on the quality of education provided by schools. 


The specific standards that each accrediting agency develops may differ. 
Accreditors may also establish standards in areas not required by law.''® 
For example, accreditors may evaluate schools based on governance, 
which helps ensure that the board, the school administration, and the 
faculty each have responsibilities according to their expertise and 
function. 


Accrsditor Sanctions and when schools do not meet accreditor standards, accrediting agencies 
Other Decisions impose sanctions or take other interim actions (such as requiring 


'"'20 U.S.C. § 1099b(a)(4)(A) and (5). 

'®20 U.S.C. § 1099b(a)(4) and 34 C.F.R. § 602.16(a). 
'®20 U.S.C. §§ 1099b(o)and 1099b(g). 

'^20U.S.C. § 1099b(g). 

'®20 U.S.C. § 1099b(g). 
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Sanction Definitions 

Regional Accrediting Agencies (as of April 

2014): 

• Termination: a school has its accreditation 
withdrawn. 

• Probation: a school has been determined 
not to meet one or more standards for 
accreditation. An indication of a serious 
concern about the level of non-compliance 
related to the standards. 

• Warning: a school has been determined 
not to meet one or more standards for 
accreditation. 

• Show cause order: a school is asked to 
demonstrate why its accreditation should 
not be withdrawn. 

National Accrediting Agencies: 

• Termination: a school has its accreditation 
withdrawn. 

• Probation: no single agreed-upon 
definition, but could be used to indicate a 
serious concern about non-compliance with 
one or more standards. 

• Show Cause Orders and Warnings: no 

single agreed-upon definition for either. 
“Warning" may indicate that a school has 
been determined not to meet one or more 
standards for accreditation. “Show cause” 
may indicate an accreditor will impose a 
sanction if a school cannot demonstrate 
why it should not be imposed. 

Source: Council of Regional Accrediting Commissions and GAO 

analysis of information from accrediting agencies. | GAO-15-59 


annual reports on finances). The Higher Education Act requires 
accreditors to report certain sanctions, including terminations and 
probations, to Education within 30 days, and to provide Education a 
summary of the reasons leading them to terminate a school’s 
accreditation.^® Regional accreditors recently agreed on common 
sanction definitions, while national accrediting agencies do not have 
agreed-upon sanction definitions (see sidebar). 

In addition to issuing sanctions when schools do not meet accreditor 
standards, accrediting agencies may take a variety of other actions, such 
as requiring annual reports on finances or enrollment, requiring an interim 
report on whether a school is meeting certain accreditor standards, or 
reaccrediting schools for a shorter amount of time (see fig. 3). Moreover, 
several of the accreditors we contacted noted that a key way they ensure 
newly applying schools meet minimum quality standards is to deny initial 
accreditation to any school that does not fully meet their standards. For 
example, national accreditors tend to have their own benchmarks to 
assess academic quality, which might include licensure exam pass rates 
for fields in which graduates must pass an exam to practice. If a school 
does not meet the standards developed by an accreditor for initial 
accreditation — including any academic quality standards, such as 
licensure exam pass rates — it could be denied initial accreditation. 


^^Accreditors must also report such sanctions, and provide summaries to the appropriate 
state licensing or authorizing agency. 20 U.S.C. § 1099b(a)(7) and (8). Specifically, 
accreditors must notify Education and the appropriate state licensing or authorizing 
agency of any final decision to place a school on probation; deny, withdraw, suspend, 
revoke, or terminate a school’s accreditation; or take other adverse action, as defined by 
the accrediting agency. 34 C.F.R. § 602.26(b). Accreditors must provide written notice to 
the public of such sanctions within 24 hours of its notice to the school. 34 C.F.R. § 
602.26(c). 
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Figure 3: Range of Potential Actions Available to Accrediting Agencies 



School seeking accrediation 

Source: GAO interviews with accrediting agencies and a review of accreditor documentation. | GAO-15-59 


Education’s Oversight of 
Schools and Accreditors 

Oversight of Schools Education oversees schools’ compliance with federal student aid 

requirements in several ways, including by monitoring their financial 
activities and conducting in-depth reviews known as program reviews. For 
example, Education calculates the financial health of for-profit and 
nonprofit schools and provides additional oversight when a school’s score 
falls below a certain level. For a program review. Education assesses a 
school’s compliance with relevant federal laws and regulations.^'' If a 
school has weak federal financial health indicators or significant findings 
on a program review. Education may take enforcement action against a 
school by requiring it to post a letter of credit to ensure it meets its 
financial obligations to the department or submit additional 


Education does not calculate financial health scores for public schools because it 
considers a public school to be financially responsible if its debts and liabilities are backed 
by the full faith and credit of the state or other government entity. 

Education conducts about 300 program reviews per year. 
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Oversight of Accreditors 


documentation, such as detailed student information, before receiving 
federal student aid funds. Education may also limit, suspend, or end the 
school’s eligibility to participate in federal student aid programs. For 
example. Education terminates access to federal student aid when a 
school loses accreditation. 

Education also collects information about student outcome characteristics 
that research has found can be related to academic quality, such as 
graduation, retention, and dropout rates, to provide information to the 
public. Education does not use these outcomes to regularly assess 
school compliance with federal student aid requirements. 

The Higher Education Act requires accrediting agencies to meet certain 
statutory recognition criteria and have certain operating procedures in 
order to be recognized by Education, and accrediting agencies must have 
their recognition renewed by Education at least every 5 years. To 
recognize an accrediting agency. Education’s Accreditation Group and 
the independent National Advisory Committee on Institutional Quality and 
Integrity (NACIQI), which advises the Secretary of Education on 
accreditation issues, must: 

• review and assess compliance with Education’s recognition criteria, 
including accrediting agency standards, policies, and procedures in 
areas required by statute (as seen in fig. 4);^'^ 

• evaluate the extent to which an agency applies these standards, 
policies, and procedures when it accredits schools; and 

• make a recommendation on whether and for how long the accreditor 
should be re-recognized to the Assistant Secretary for Postsecondary 
Education, or other official who is designated to make the final 
decision regarding recognition. 


2^20 U.S.C. § 1099b(a), (c), and (d). 

NACIQI advises the Secretary of Education on matters related to postsecondary 
accreditation and the eligibility and certification process for postsecondary schools to 
participate in federal student aid programs. The House of Representatives, the Senate, 
and Education each appoint six of NACIQI’s 18 members. NACIQI members are 
appointed on the basis of their technical qualifications, professional standing, and 
demonstrated knowledge in the fields of accreditation and administration in higher 
education. 20 U.S.C. § 1011c. 

^^See 20 U.S.C. § 1099b. 
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If Education has concerns regarding an agency’s standards or application 
of them, the Assistant Secretary may choose to recognize the accrediting 
agency for a limited time, or reject the agency’s application for recognition 
or re-recognition. Education is required to publish a list of accrediting 
agencies that the Secretary recognizes as reliable authorities on the 
quality of education or training provided by the schools they accredit.^® 


Figure 4: Education’s Process for Recognizing Accrediting Agencies 


Recognition decisions last for up to 5 years, 
depending on Education’s findings 


B Education’s recognition decision 

Assistant Secretary for Postsecondary 
Education reviews the recommendations 
from Education and NACiQi before 
making a finai decision on the agency’s 
recognition status. 



□ Accrediting agency seif-evaiuation° 

Accrediting agency submits an appiication for recognition that 
provides evidence of the agency's compiiance with Education's 
recognition criteria, aiong with supporting documentation. 


B Education staff anaiysis 
and recommendation 

Education reviews the 
accrediting agency's standards, 
poiicies, and procedures for 
assessing schoois and 
conducts a site visit before 
making a recommendation. 


Q NACIQI recommendation 

NACiQi makes a recommendation 
to the Assistant Secretary for 
Postsecondary Education based 
on its assessment. 


Source: GAO analysis of information from the Department of Education and relevant federal law and regulations. | GAO-15-59 


E] NACIQI review and public meeting 

National Advisory Committee on institutionai 
Quaiity and integrity (NACiQip conducts a 
second assessment of the accrediting agency, 
relying in part on some of Education's findings 
and documentation. NACiQi hoids pubiic 
meetings generaiiy twice a year to review 
appiications for recognition. 


^Recognition Process: 34 C.F.R. Part 602, Subpart C. 

‘’NACiQi advises the Secretary of Education on matters reiated to postsecondary accreditation and 
the eiigibiiity and certification process for postsecondary schoois to participate in federai student aid 
programs. The House of Representatives, the Senate, and Education each appoint six of NACiQI’s 
18 members. NACiQi members are appointed on the basis of their technicai quaiifications, 
professionai standing, and demonstrated knowiedge in the fieids of accreditation and administration 
in higher education. 20 U.S.C. § 1011c. 


2^20 U.S.C. § 1001(C). 
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Accreditors 
Terminated the 
Accreditation of Less 
Than 1 Percent of 
Their Member 
Schools 


Accreditors Rarely 
Terminated Accreditation, 
and More Frequently 
Issued Other Sanctions 
against Schools 


From October 2009 through March 2014, accreditors issued at least 984 
sanctions against 621 schools — about 8 percent of member schools.^® 
During this timeframe, accreditors terminated the accreditation of less 
than 1 percent of their member schools (66 schools) and placed about 4 
percent (328 schools) on probation. Accreditors sanctioned some 
schools more than once during our timeframe. For example, 45 of the 66 
schools (68%) whose accreditation was terminated had received at least 
one prior sanction since the start of fiscal year 201 0.^® In addition, 74 of 
the 328 schools (23%) with a probation sanction received at least one 
other sanction during our timeframe. See figure 5 for information on the 
number and type of sanctions issued by fiscal year. 


^®We limited our review to accrediting agencies that accredit schools, including 
programmatic accreditors that accredit free-standing schools, as opposed to those that 
accredit programs within schools. We included all probations and terminations (both are 
required to be reported to Education), as well as show cause orders and warnings that 
accreditors voluntarily reported to Education. It is difficult to know how many schools were 
placed on show cause or warning overall because accreditors are not required to report 
these data to Education. We requested additional information on show cause orders and 
warning sanctions against member schools from accreditors that did not report these data 
to Education for fiscal years 2010, 2011, 2012, and 2013, and received responses from 27 
of 31 accreditors. Based on these responses, we added an additional 90 show cause 
orders and 26 warning sanctions to our data set. We did not include sanctions that were 
continued, but not started, during our time frame. Overall, 8,187 schools were accredited 
at the time of our review, including both schools that were eligible to participate in federal 
student aid programs and those that were not eligible or that were eligible but not drawing 
down federal funds. These schools may have been receiving other forms of federal aid 
such as grants. 

According to our analysis, accreditors also issued show cause orders to at least 1 .5 
percent of member schools and warnings to over 2 percent. 

oo 

Of these, 34 had a prior probation sanction, nine had a prior show cause order, one had 
a prior show cause order and probation sanction, and one had prior warning and probation 
sanctions. 
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Figure 5: Number of Schools with Sanctions by Type of Sanction and Fiscal Year, 
October 2009 through March 2014 

Number of schools with sanctions 
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Source: GAO analysis of data from the Department of Education’s accreditation database and followup with accreditors. 


I GAO-15-59 


Notes: Any schooi couid have received a sanction in more than one fiscai year and couid have 
received more than one sanction within a given fiscai year, if a schooi was issued the same type of 
sanction more than once in a fiscai year, it is counted once for that type of sanction in this figure. The 
show cause orders and warnings depicted by this graph were voiuntariiy reported to Education or 
obtained from accreditors by GAO. Uniike terminations and probations, show cause orders and 
warnings are not required to be reported to Education. Show cause orders may indicate an accreditor 
wiii withdraw accreditation if a schooi cannot demonstrate why it shouid not be terminated. 

Data for federai fiscai year 2014 cover the first haif of the year — October 2013 through March 2014 — 
which were the most recent avaiiabie data at the time of our study. 


In addition, the proportion of member schools that accreditors sanctioned 
varied. For example, two accreditors each sanctioned fewer than 2 
percent of their member schools during our timeframe, compared to 41 
percent for another accreditor. A representative from one accrediting 
agency explained that a key challenge for accreditors is grappling with 
competing expectations of accreditation. The representative noted that 
there is a general view by policy makers and those who influence policy 
that accreditors do not terminate accreditation enough. However, if an 
accreditor does terminate a particular school’s accreditation, she said 
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there may be significant negative reaction from the public in the affected 
region, and a view that the accreditor is being too punitive.^® 


Accreditors Sanctioned 
For-Profit, Public, and 
Nonprofit Schools 

Sanctions by School Accreditors issued sanctions to for-profit, public, and nonprofit schools 

Characteristics various program lengths and enrollment. Accreditors issued 

terminations and probations mainly to for-profit schools whose longest 
programs were 2 years or less, and which enrolled fewer than 1 ,000 
students (see fig. 6).®° Over 80 percent of terminated schools and 63 
percent of schools placed on probation were for-profits, even though for- 
profit schools make up only about 38 percent of all schools participating in 
federal student aid programs.®^ 

In contrast, accreditors mainly issued warnings to public schools whose 
longest programs were 2 years or nonprofit schools whose longest 
programs were 4 years or longer, both of which enrolled fewer than 5,000 
students. Almost half of the 186 schools that received warning sanctions 
were public, although public schools make up about 32 percent of all 
schools participating in federal student aid programs (see fig. 6). 


Of the 38 accreditors we examined, over half (1 9) did not issue any formal sanctions 
during our timeframe. According to the accreditation database, these 19 agencies are 
generally programmatic accreditors that accredit about 5.6 percent of all of the accredited 
schools. They tend to accredit schools that are small and offer only one type of program 
(e.g., nursing programs). 

®°We focused on terminations and probations because accreditors are required to report 
these sanctions to Education. 

o-| 

207 of the 327 schools with probations were for-profit schools and 55 of the 66 schools 
with terminations were for-profit schools. Show cause orders were also mainly issued to 
for-profit schools whose longest programs were 2 years or less and which enrolled fewer 
than 1,000 students. 
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Figure 6: Sanctions against Schoois (percent) by Schooi Type, October 2009 through March 2014 
Rate of sanctions 
100 


Public 

Nonprofit 

For-profit 


Source: GAO analysis of data from the Department of Education’s Integrated Postsecondary Education Data System (IPEDS) database. | GAO-15-59 

Note: Percentages may not add to 100 because of rounding. The show cause orders and warnings 
depicted by this graph were voluntarily reported to Education or obtained from accreditors by GAO. 
Unlike terminations and probations, show cause orders and warnings are not required to be reported 
to Education. Show cause orders may indicate an accreditor will withdraw accreditation if a school 
cannot demonstrate why it should not be terminated. 

There were also variations in some of the student characteristics at 
schools with different types of sanctions. For example, terminated schools 
had fewer students and a smaller proportion of part-time students 
compared to schools with probations, show cause orders, and warning 
sanctions and to all schools participating in federal student aid programs. 
Terminated schools also had a larger proportion of students receiving Pell 
grants compared to schools issued other sanctions, and to all other 
schools. In contrast, schools with warnings tended to have more students, 
a higher proportion of part-time students, and a lower proportion of 
students receiving Pell grants, compared to schools receiving other 


83.3 
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sanctions, and to all schools participating in federal student aid 
programs. 

Sanctions by Type of National accreditors and regional accreditors each sanctioned about 8 

Accreditor percent of their member schools during our timeframe. National 

accreditors were more likely to issue probations and regional accreditors 
were more likely to issue warnings compared to other types of sanctions. 


In Fiscal Year 2012, 

Accreditors Most 
Frequently Issued 
Terminations or Probations 
to Schools that Did Not 
Meet Accreditor Standards 
on Financial Capability 

In fiscal year 2012, accreditors most frequently issued terminations and 
probations to schools for not meeting accreditation standards on financial 
capability (62 and 58 percent of these sanctions, respectively) — such as 
not appropriately demonstrating that the financial structure of the school 
was sound. 


Reasons for Accreditor 
Sanctions 


exception to this was schools with probations, which had a higher proportion of part- 
time students than schools with warnings. Pell grants primarily benefit low-income 
students, indicating that schools that received warnings, as opposed to terminations, had 
a higher-income student body. GAO, Higher Education: improved Tax information Couid 
Heip Famines Pay for Coiiege, GAO-12-560 (Washington, D.C.: May 18, 2012). 

^^We reviewed fiscal year 2012 sanctions because this was the most recent full year of 
data available when we began our detailed review of all accreditor notification letters to 
schools to determine the reason(s) for each sanction. Our analysis of all terminations and 
probations is based on 26 schools that had their accreditation revoked and 101 schools 
that were placed on probation in fiscal year 2012. Five of these schools received both a 
probation and a termination during this time. See appendix I for a detailed discussion of 
our methodology. Accreditor standards for academic quality and financial capability are 
different from the information collected by Education on student outcome and financial 
characteristics. 

^■^In the sample of 10 show cause orders and 10 warnings that we reviewed, show cause 
orders were also most frequently issued for reasons of financial capability. In contrast, the 
warnings were most frequently issued for academic quality reasons — such as failing to 
comply with educational objectives or to measure student learning outcomes. Accreditors 
also frequently issued probations for reasons related to academic quality and governance. 
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Reasons for Accreditor Sanctions 
Academic quality: issues with student 
achievement in reiation to the mission and 
curricuia, or other student outcomes. 
Administrative capability: issues such as 
those reiated to faciiities, suppiies, and 
administrative capabiiity. 

Financial capability: issues with financiai 
capabiiity and compiiance with federai student 
aid responsibiiities. 

Integrity: fraud or misrepresentation. 
Governance: issues with division of 
responsibiiity, such as between the Board and 
a coiiege president. 

Institutional Effectiveness: issues reiated to 
iong-term pians for assessing iearning and 
academic achievement. 

Other: other issues that do not faii into the 
categories noted above. 

Source: GAO analysis of information from accreditors and 
Education. | GAO-15-59 


See figure 7 for the number of terminations and probations issued for 
each type of reason. According to an accreditor we spoke with, regional 
accreditors are more likely than national accreditors to have a 
governance standard, to ensure the appropriate division of responsibility 
between the Board and faculty at public schools. Refer to side-bar for 
additional information about the reasons for accreditor actions. 
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Figure 7: Number of Terminations and Probations Issued by Reason for Sanctions, Based on GAO’s Review of Accreditor 
Standards and Reasons Analysis, Fiscal Year 2012 
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Source: GAO analysis of data from the Department of Education’s accreditation database and from accreditor notification letters. | GAO-1 5-59 


Note: This figure includes all terminations and probations taken during fiscal year 2012. The count of 
the reasons for sanctions does not add up to the total number of sanctions because sanctions could 
be taken for more than one reason. Of the 127 terminations and probations we reviewed, 44 were 
taken for multiple reasons and we counted each reason (e.g., 3 sanctions were taken for 6 reasons 
each). The reasons are based on our review of accreditor standard areas. Accreditor standards for 
assessing academic quality and financial capability are different from the information collected by 
Education on student outcome and financial characteristics. 


Reasons by Type of Accreditor The reasons cited for terminations and probations varied by type of 

accreditor. Consistent with our overall analysis of reasons across all 
types of accreditors, national accreditors more frequently issued 
sanctions to schools that did not meet accreditors’ financial capability 
standards, compared to other reasons (see fig. 8).^® For example, one 
national accreditor terminated the accreditation of a school because it did 
not appropriately demonstrate that its financial structure was sound. In 


®®We only included terminations and probations in this analysis, since we did not have the 
full universe of show cause orders and warnings, which accreditors are not required to 
report to Education. 

®®Programmatic accreditors issued most of their sanctions for reasons of academic quality 
or administrative or financial capability. 
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contrast, regional accreditors most frequently issued sanctions to schools 
that did not meet accreditors’ academic quality capability standards, 
followed by financial and administrative capability. 


Figure 8: Number of Terminations and Probations Issued by Type of Accreditor and Reason for Sanction Based on GAO’s 
Review of Accreditor Standards and Reasons Analysis, Fiscal Year 2012 
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Source: GAO analysis of data from the Department of Education’s accreditation database and from accreditor notification letters. | GAO-1 5-59 


Note: Any sanction could have been taken for more than one reason. This figure excludes 
terminations and probations taken by programmatic accreditors. At the time of our review, regional 
accreditors had 3,134 member schools in total, and national accreditors had 3,719. Accreditor 
standards for assessing academic quality and financial capability are different from the information 
collected by Education on student outcome and financial characteristics. 


Reasons by School Reasons for accreditor sanctions varied by the type of accredited school. 

Characteristics For-profit schools received the highest proportion of sanctions for reasons 

related to financial capability (43 of 104) from their accreditors. In 
contrast, public and nonprofit schools received the highest proportion of 
sanctions for academic quality reasons (19 of 47 and 12 of 43) (see fig. 

9). 37 


37Qur analysis of school and student characteristics includes all of the schools in the 
reasons analysis (142) that we were able to link to IPEDS for each given variable (the 
number of schools for a given variable ranged from 99 to 119). As a result, the universe 
for schools with sanctions for each variable is based on the number of schools that had a 
value for that variable. A single sanction can be issued for more than one reason. 
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Figure 9: Number of Terminations and Probations Issued by Type of School and Reason for Sanction, Fiscal Year 2012 
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Source; GAO analysis of data from the Department of Education’s accreditation database, accreditor notification letters, and from the Integrated Postsecondary Education Data System. | GAO-15-59 


Note: The “other” category covers reasons that did not fit into the existing categories, such as findings 
by the accreditor that the school was not following applicable federal or state laws. 


Reasons for accreditor sanctions also varied for combinations of school 
type and program length. For example, for-profit schools with programs of 
2 years or less were most frequently sanctioned for reasons related to 
financial capability, while nonprofit schools with programs of 2 years or 
less were most frequently cited for academic quality. Accreditors also 
most frequently cited academic quality reasons for for-profit schools with 
programs of 4 years or longer, but at nonprofit schools with programs that 
lasted more than 4 years, accreditors most frequently cited financial 
capability issues. All public schools, regardless of size, were most 
frequently cited for academic quality as the reason for sanctions. 

Financial capability may be less of an issue for public schools because 
they receive state and local funding. 

See appendix II for additional data on selected characteristics of schools 
and the students at schools with sanctions. 
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Accreditors Were No 
More Likely to 
Sanction Schools with 
Weaker Student 
Outcomes Than 
Those with Stronger 
Outcome 
Characteristics 

We found that, on average, accreditors were no more likely to issue 
terminations or probations to schools with weaker student outcomes 
compared to schools with stronger student outcomes from October 2009 
through March 2014, as seen in table 2 below. This held true for one 
combined indicator incorporating all of the student outcome 
characteristics we reviewed, as well as for most of the individual 
characteristics we examined. (The sidebar describes the student outcome 
characteristics we examined.)^® Regional accreditors, however, were 
more likely to issue terminations or probations to schools with weaker 
outcomes on the combined indicator. (See appendix I for additional 
details on this analysis and appendix III for additional information on 
accreditor sanctions associated with student outcomes.) 


Overall, Accreditors Were 
No More Likely to Issue 
Terminations or Probations 
to Schools with Weaker 
Student Outcomes Than 
Those with Stronger 
Outcomes 


oo 

Because program review findings could reflect either student outcome or financial 
characteristics of schools, we included this variable in both the student outcome and 
financial characteristics analyses. 
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Selected Student Outcome Characteristics: 

• Three-Year Cohort Default Rate: the 

percent of borrowers in default 3 years after 
entering repayment status. Education views 
this characteristic as an indicator of academic 
quality at schools, since students who 
received a lower quality education may be 
less likely to have adequate income to repay 
their loans. 

• Forbearance Rate: the percent of borrowers 
in forbearance (and therefore not repaying 
their loans on a temporary basis) during the 
official cohort default period. Education views 
this characteristic as an indicator of academic 
quality at schools, since students who 
received a lower quality education may be 
less likely to have adequate income to repay 
their loans. 

• Graduation Rate: the percent of first-time 
full-time degree/certificate-seeking 
undergraduate students who complete a 
program within 1 50 percent of the program 
length. A low graduation rate may indicate a 
lack of academic quality. 

• Dropout Rate: the percent of students who 
left school during a particular year, but did not 
graduate. A high dropout rate may indicate a 
lack of academic quality. 

• Retention Rate: the percent of first-time 
degree/certificate-seeking students who 
enrolled in one fall and either successfully 
completed their program or re-enrolled in the 
next fall. A low retention rate may indicate a 
lack of academic quality. 

• Increases in Federal Student Aid: annual 
growth in federal student aid volume, which 
may indicate in extreme cases that growth 
may be too rapid to maintain academic and 
administrative services needed to adequately 
support students. 

• Number of Program Review Findings: the 

number of findings at schools selected by 
Education for in-depth review due to the 
presence of certain risk factors, and the 
number of issues found in those reviews. 

Source: GAO analysis of information on selected student 

characteristics coilected by Education. | GAO-15-59 


Table 2: Likelihood of Termination or Probation for Schools with Weaker vs. 
Stronger Overall Student Outcome Characteristics, by Type of Accreditor, October 
2009 through March 2014 


Was there a significant difference in accreditors’ responses to weaker and 
stronger student outcomes at schools? 



Nonprofit and for-profit 



schools 

Public schools 

All Accreditors 

No 

No 

Regional accreditors 

Yes 

Yes 

National accreditors 

No 

NA® 


Source: GAO anaiysis of schooi-ievel student outcome characteristics coilected by Education and data from the accreditation database. 

I GAO-15-59 

Notes: We used statistical techniques that allowed us to concurrently examine accreditors’ likelihood 
of sanctioning schools with weaker versus stronger student outcomes across one combined indicator 
incorporating a range of related student outcome characteristics. ”Yes” indicates a statistically 
significant difference in the likelihood of termination or probation between schools in the top and 
bottom 25 percent of the distribution of combined scores. For regional accreditors of nonprofit and for- 
profit schools, the difference was significant at the 90 percent confidence level, while the other 
differences were all significant at the 95 percent confidence level. 

^Because of sample size limitations, the results of for-profit and nonprofit schools could not be 
separated and we could not analyze nationally accredited public schools as its own group. 


We also examined how likely accreditors were to issue terminations or 
probations to schools with weaker student outcomes for each individual 
characteristic we considered. The results of this outcome-by-outcome 
analysis also suggest that, on average, accreditors were not more likely 
to issue terminations or probations to schools with weaker student 
outcome characteristics collected by Education than those with stronger 
outcomes, with the exception of default rates (see table 3). For instance, 
national accreditors did not issue terminations or probations to schools 
with low graduation rates at a higher rate than schools with high 
graduation rates. Regional accreditors, on the other hand, were more 
likely to do so for most of the student outcome characteristics we 
examined, as seen in table 3. Accounting for relevant differences across 
schools’ student bodies-such as the percentage of students who were 
low-income. Black, Hispanic, over the age of 25, or attending school part 
time — did not affect the association between each outcome we analyzed 
and the chance of a termination or probation. 

Although accreditors are required by law to have standards in academic 
and financial areas, among others, they are not required to use the 
student outcome characteristics that we selected to assess school 
academic quality, or to sanction members with weaker outcomes. Some 
accreditors do examine school student-level outcomes as benchmarks to 
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determine whether their member schools are providing quality education, 
but would not necessarily sanction or revoke the accreditation of a school 
for not meeting these benchmarks.^® 


Table 3: Likelihood of Termination or Probation for Schools with Weaker vs. Stronger Individual Student Outcome 
Characteristics, by Type of Accreditor, October 2009 through March 2014 


Was there a significant difference in accreditors’ responses to weaker and 
stronger student outcomes^ at schools? 

Forbearance 



Default Rate 

Graduation Rate 

Dropout Rate 

Retention Rate 

Rate 

Overall 

Yes 

No 

No 

No 

No 

Regional accreditors 

Yes 

Yes 

Yes 

Yes 

No 

National accreditors 

No 

No 

No 

No 

No 


Source: GAO analysis of school-level student outcome characteristics collected by Education and data from the accreditation database. | GAO-1 5-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker student outcome characteristics, compared to schools with stronger 
outcomes, for each individual outcome. Schools with weaker student outcomes were considered to 
be those in the bottom vs. the top for each characteristic (those in the 1st vs. 99th percentile and 5th 
vs. 95th percentile). “Yes” indicates that the difference between the 1st and 99th percentiles and/or 
5th and 95th percentiles was statistically significant at the 95 percent confidence level. All 
comparisons were significant for the 1st and 99th percentiles as well as for the 5th and 95th 
percentiles, with the exception of default rate for regional accreditors, which was only significant when 
comparing the 5th and 95th percentiles. 

’’Default rate indicates the percent of borrowers who entered repayment in fiscal 2009 or 2010 and 
were in default as of the end of the second following fiscal year; graduation rates reported to IPEDS 
in 201 1 and 2012 are for first-time full-time degree/certificate-seeking undergraduate students that 
completed their degree within 1 50 percent of the expected time; dropout rate indicates the total 
number of withdrawals reported by each school during a particular year divided by the total number of 
graduates plus withdrawals reported to the National Student Loan Data System for that year for 
award years 2008-2009 through 2012-2013; retention rate indicates the percent of first-time 
degree/certificate-seeking students who enrolled in the previous fall and either successfully 
completed their program or re-enrolled in the next fall as reported to IPEDS in the fall of 2010 and 
201 1 ; and forbearance rate indicates the percent of borrowers who entered repayment status in fiscal 
year 2009 and 2010 and were in forbearance as of the end of the following fiscal year. 

Because the graduation rate collected by Education is limited to first-time 
full-time degree/certificate-seeking undergraduate students, we also 
estimated accreditors’ likelihood of sanctioning schools with higher 


®®For example, one accreditor noted that it takes an array of monitoring and reporting 
actions and sanctions with respect to below-benchmark student outcomes, and that it 
allows institutions an opportunity to demonstrate the success of its students in a program 
with below benchmark rates based on various factors such as the school’s ability to 
remedy its deficiencies. 
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dropout rates. Similar to the results of our graduation rate analysis, we 
found that national accreditors were not more likely to issue terminations 
or probations to schools with higher dropout rates than those with lower 
dropout rates. In contrast, regional accreditors were more likely to issue 
terminations or probations to schools with higher dropout rates (see table 
3 above). 

Our findings suggest that terminations and probations issued by national 
accreditors may not focus on schools with weaker student outcomes that 
are collected by Education, although national accreditors often establish 
their own quantitative student outcome standards for member schools. 
Conversely, while regional accreditors typically rely on member schools to 
determine their own standards of academic quality rather than 
establishing quantitative student outcome standards, their terminations 
and probations appear to more consistently focus on schools with weaker 
student outcomes. Accreditors are not required to use information on 
student outcomes collected by Education to assess the academic quality 
of schools, and no single outcome can be used to fully examine 
something as complex as educational quality. However, prior research 
and experts in the field of postsecondary education have reported that 
assessing multiple student outcomes (e.g., graduation rates and student 
loan default rates) could shed light on the quality of education provided by 
schools. 

There are various factors that may help account for our findings: 

• Limitations in currently available information on student outcome 
characteristics at schools, which make certain characteristics less 


^^Specifically, Education excludes part-time and returning students from its graduation 
rate calculations. Therefore, these calculations may not adequately reflect student 
outcomes at schools serving a high proportion of non-traditional students. Our dropout 
rate calculation includes the total number of withdrawals reported by each school during a 
particular award year divided by the total number of graduates plus withdrawals reported 
for that year. The dropout rate includes both part-time and returning students, and 
excludes students who do not receive federal student aid. 

^^For example, one major national accrediting agency requires each program at 
accredited schools to have graduation and job placement rates not less than one standard 
deviation below the mean for comparable programs. 

^^While accreditors may use school-developed standards to demonstrate student 
success. Education requires accreditors using such standards to have mechanisms in 
place to assess those standards. 
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useful in evaluating the academic quality at some types of schools. 

For instance, community colleges typically serve a high proportion of 
part-time and returning students, but these students are not included 
in official graduation rate statistics. In addition, accreditors may review 
different outcomes, such as job placement rates, that are not captured 
by Education. Further, some accreditors require schools to have 
their own processes in place to assess student outcomes. In general, 
accreditors noted more challenges in monitoring schools’ academic 
quality than their financial stability due to limitations in currently 
available information on academic quality. 

• The difficulty of comparing actions other than sanctions. For example, 
accreditors may provide oversight in other ways than issuing 
sanctions, such as conducting frequent monitoring. An official from an 
association representing accrediting agencies noted that one 
explanation for a low rate of sanction taking may be that some 
accreditors would work with a school to improve the quality of its 
academic programs rather than apply sanctions. In addition, a 
representative from one regional accreditor noted that its oversight of 
currently accredited member schools seeking reaffirmation of their 
accreditation is based on school-wide information and analysis that 
goes beyond a set of outcomes and they are aware of the contextual 
details that would allow them to determine if a sanction is needed. 

See appendix III for additional findings on how the likelihood of probation 
or termination is associated with selected student outcome characteristics 
at schools. 

Accreditors are required under the Higher Education Act to consider 
student achievement in their assessments of academic quality, but they 
do not have to use information collected by Education on student 
outcomes to do so, and the information they use may differ from 
Education’s.'^'^ The ways accreditors assess student outcomes and 


'^^For information on state licensing and job placement rates by school sector, see GAO, 
Postsecondary Education: Student Outcomes Vary at For-Profit, Nonprofit, and Pubiic 
Schoois, GAO-12-143 (Washington, D.C.: December 2011). 

'^'^The Higher Education Act specifically requires accreditors to assess schools’ success 
with respect to student achievement in relation to the school’s mission. These 
assessments may include different standards for different schools or programs, as 
established by the school, including, as appropriate, consideration of course completion, 
state licensing examination, and job placement rates. 20 U.S.C. § 1099b(a)(5)(A). 
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academic quality vary widely/® For example, some accreditors use 
graduation rates — one of the student characteristics of schools that is 
collected by Education — while others have more qualitative standards 
such as whether a school is evaluating success with respect to its 
mission. However, in order for an accreditor to be recognized. Education 
must determine that its standards are effective in ensuring that the 
education schools offer is of sufficient quality to achieve the stated 
objective."^® 

In addition, accreditors we spoke with said that their goal for schools is 
the same as Education’s — to ensure student achievement."^^ While the 
federally available information on student outcomes that we used in our 
analysis has limitations and may vary from what is used by accreditors, it 
nonetheless provides important information about student achievement 
among accreditors’ member schools. 

Our analysis raises questions about whether the standards accreditors 
currently use ensure that schools provide a quality education, and 
whether Education is effectively determining if these standards ensure 
educational quality.. Consequently, schools that do not provide a quality 
education may be allowed access to federal student aid funds. Federal 
internal control standards require agencies to assess risk, including risk 
from external parties, and establish procedures to safeguard federal 
funds. Identifying ways to strengthen their assessment of the 
effectiveness of accreditor standards could help Education to protect the 
billions of dollars of federal student aid funds that the department 
distributes each year. 


Accreditor standards may vary in part because they reflect the diverse missions of the 
schools that accreditors oversee, which can range from small vocational schools to large 
research universities. 

"^^20 U.S.C. § 1099b(a)(4)(A) and 34 C.F.R. § 602.16(a). 

"^^One accreditor also noted that accreditors have additional goals, such as ensuring the 
institutions operate with integrity and have the resources needed to ensure stability and 
provide a quality education. 

"^®GAO/AIMD-00-21.3.1. 
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Overall, Accreditors Were 
More Likely to Issue 
Terminations or Probations 
to Schools with Weaker 
Financial Characteristics 
than to Their Peers with 
Stronger Characteristics 


Selected Financial Characteristics: 

• Financial Health Scores: annual Education 
calculation of the financial health of for-profit 
and nonprofit schools using three ratios that 
capture different aspects of a school’s overall 
financial health. 

• Program Reviews: schools selected by 
Education for in-depth review due to the 
presence of certain risk factors, many of 
which are financial, and the number of issues 
found in those reviews. 

• Other Education Sanctions: schools 
sanctioned by Education for issues regarding 
the administration of federal student aid 
funds, which may relate to financial health 
and problems with a school’s financial 
stability. Sanctions may include heightened 
cash monitoring (requiring schools to 
disburse federal student aid funds before 
receiving funds) or a letter of credit (an 
assurance that financial obligations to 
Education would be met in the event of 
school closure). 

• Decreases in Federal Student Aid: sharp 
declines in annual federal student aid volume 
may indicate financial instability for a school. 

• 90/10 Ratio: for-profit schools must report the 
percentage of their total revenues obtained 
from federal student aid funds (referred to as 
the 90/10 rate). A high 90/10 ratio indicates 
high reliance on federal student aid funds and 
risk of non-compliance with a requirement 
that a for-profit school must obtain at least 10 
percent of its revenue from non-Title IV 
sources. This could also indicate that a 
school is having difficulty obtaining other 
sources of funds and financial instability. 

Source: GAO analysis of information on selected financial 

characteristics collected by Education. | GAO-15-59 


We found that, on average, accreditors were more likely to issue 
terminations or probations to schools with weaker financial characteristics 
compared to schools with stronger characteristics from October 2009 
through March 2014 (see table 4). National and regional accreditors were 
also more likely to sanction schools with weaker financial characteristics 
than those with stronger ones on our one combined indicator for financial 
responsibility (see sidebar for a description of the characteristics 
examined). (See appendix I for additional details on this analysis and 
appendix IV for additional findings on how accreditor sanctions are 
associated with the financial characteristics of schools.) 


Table 4: Likelihood of Termination or Probation for Schools with Weaker vs. 
Stronger Overall Financial Characteristics, by Type of Accreditor, October 2009 
through March 2014 


Was there a significant difference in accreditors’ responses to weaker and 
stronger financial characteristics at schools? 

All accreditors Yes 

Regional accreditors Yes 

National accreditors Yes 


Source: GAO analysis of financial characteristics information collected by Education and data from the accreditation database. | 
GAO-15-59 

Notes: We used statistical techniques that allowed us concurrently to examine accreditors’ likelihood 
of sanctioning schools with weaker versus stronger financial characteristics across one combined 
indicator incorporating a range of related financial characteristics. “Yes” indicates a statistically 
significant difference at the 95th percent confidence level in the likelihood of termination or probation 
between schools in the top and bottom 25th percent of the distribution of combined scores. 

This analysis is restricted to nonprofit and for-profit schools. Because Education does not measure 
the financial health of public schools, as they rely, in large part, on state and local funding, we did not 
include them in our review of this characteristic. 


Our analysis produced similar findings when we separately examined 
accreditors’ likelihood of sanctioning schools with weaker individual 
financial characteristics for each characteristic we considered. For 
instance, both national and regional accreditors were more likely to 


^®We used the same statistical technique to compare the results for schools across a 
range of related financial characteristics concurrently as we used to compare student 
characteristics. We analyzed whether or not a school received a termination or probation 
from at least one accreditor from October 1, 2009 through March 31, 2014. Because 
Education does not measure the financial health of public schools, as they rely, in large 
part, on state and local funding, we did not include them in our review of this 
characteristic. 
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sanction member schools with low financial health scores than those with 
high scores, as seen in table 5 below. 


Table 5: Likelihood of Termination or Probation for Schools with Weaker vs. Stronger Individual Financial Characteristics, by 
Type of Accreditor, October 2009 through March 2014 



Was there a significant difference in accreditors’ responses to weaker and stronger financial 

characteristics^ at schools? 


Financial Health 
Score*’ 

Program review by 
Education 

Number of Education 
program review findings 
related to financial health 

Sanctioned by Education 
for financial risk 

Overall 

Yes 

Yes 

Yes 

Yes 

Regional 

accreditors 

Yes 

Yes 

Yes 

Yes 

National 

accreditors 

Yes 

Yes 

Yes 

Yes 


Source: GAO analysis of financial characteristics information collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker individual financial characteristics, compared to schools with 
stronger characteristics. Schools with weaker financial characteristics were considered to be those in 
the bottom vs. the top for each characteristic (those in the 1st vs. 99th percentile and 5th vs. 95th 
percentile). “Yes” indicates that the difference between the 1st and 99th percentiles and 5th and 95th 
percentiles was statistically significant at the 95 percent confidence level. 

^Financial health score refers to a school’s fiscal year 201 0 and 201 1 financial composite score, 
which is calculated by Education using three key financial ratios measuring different aspects of a 
school’s financial health; program review by Education refers to whether a school had at least one 
program review initiated in fiscal years 201 1 or 2012 by Education; and sanctioned by Education 
indicates that a school was on heightened cash monitoring (requiring schools to disburse federal 
student aid funds before receiving funds), was required to have a letter of credit (an assurance that 
financial obligations to Education would be met in the event of a school closure), and/or had a 
program review with at least one finding in fiscal years 201 1 or 2012. 

‘’Because Education does not measure the financial health of public schools, as they rely, in large 
part, on state and local funding, we did not include them in our review of this characteristic. 


Our findings suggest that both national and regional accreditors focused 
their sanctions on schools with weaker financial characteristics. The 
challenges associated with making judgments about academic quality, 
described above, may exist to a lesser extent in assessing schools’ 
financial health, due to the availability of more widely accepted indicators 
of financial health, and the difficulties associated with assessing 


®°There were no significant differences in sanctions by accreditors based on a school’s 
outcome on the 90/10 ratio. For changes in federal student aid, schools with the biggest 
and smallest increases were more likely to be sanctioned by accreditors than those in the 
middle (99th and 95th percentile vs. 50th percentile and 1st and 5th percentile vs. 50th 
percentile, respectively). 
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academic quality, as reported by accreditors we spoke with. In addition, 
our findings suggest that accreditors may be more likely to accredit 
schools that have stronger financial characteristics, even if they have 
weaker student outcomes. Representatives of both regional and national 
accreditors noted that they had strong, regular procedures for assessing 
schools’ financial health, such as reviewing schools’ annual financial 
statements and requesting additional reporting from schools with 
problematic financial conditions. For instance, representatives of one 
national accreditor noted that they have a full-time financial analyst on 
staff who examines schools’ financial stability, and a financial review 
committee comprised of various financial experts who may require 
financial improvement plans for schools with problematic financial 
situations. They also noted that they place schools identified by Education 
as having financial problems on a watch list. Another accreditor described 
routine financial monitoring activities including reviewing annual audited 
financial statements and “red flags” that may trigger additional monitoring 
and reporting requirements, such as interim reports or site visits to 
schools. These include declining cash flow or financial health scores, 
among other information. 

See appendix IV for additional findings on how the likelihood of a 
probation or termination is associated with various financial 
characteristics at schools. 


Education Does Not 
Systematically Use 
Information on 
Accreditor Sanctions 
to Oversee Schools 
and Accrediting 
Agencies 

Education staff review accreditor sanction information and record their 
response to sanctions on an inconsistent basis, and existing guidance for 
Education staff on how to respond is unclear. Accreditor sanctions are an 
important information source about accreditor oversight of schools 
participating in federal student aid programs. Furthermore, the sanctions 


Education Staff Do Not 
Consistently Review and 
Respond to Accreditor 
Sanctions 
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Inconsistent Review of 
Sanction Information 


may provide information on potential noncompliance with federal student 
aid requirements. 

According to Education officials, Education directors who oversee schools 
typically forward copies of accreditor sanctions to their staff who are 
responsible for assuring that schools comply with federal student aid 
requirements. However, based on our review of a sample of 10 accreditor 
sanctions from fiscal year 2012 and interviews with Education officials 
who oversee schools, we found that Education did not consistently 
provide its analysts with these sanction notices.^'' In three of our 10 
selected cases. Education had received a copy of the sanction from the 
accreditor, but officials had no record that the analyst responsible for 
providing oversight of the relevant school ever received it.®^ The three 
sanctions cited failure to comply with accreditor standards in numerous 
areas including, respectively, financial practices and management; school 
integrity and quality of administrative officers; and financial resources, 
financial stability, and federal student aid program responsibilities. These 
issues could warrant further review from Education staff who oversee 
schools. For example, a school whose administrative officers do not meet 
accreditor standards may have trouble overseeing federal student aid 
funds. Education collects and archives sanction information it receives 
from accreditors in its accreditation database, and also maintains some 
accreditor sanction information in two other databases — an archival 
database and one related to grants management. However, Education 
officials we interviewed who oversee schools told us they do not access 
sanction information from these databases, which serve purposes not 
related to the ongoing oversight of schools participating in federal student 
aid programs. Analysts who oversee schools generally rely instead on 
their own working database, PEPS, for oversight information, but sanction 


^^We selected these sanctions in part because they indicated potential federal student aid 
concerns. While in three cases there was no indication of monitoring by Education, 
records showed that Education took monitoring or enforcement action in the other seven 
cases. 

^^In the other seven cases, Education was able to provide us with documentation 
indicating that the analyst who oversaw those schools had received information about the 
respective accreditor sanction. 
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Inconsistent Recording of 
Sanction Review 


information is not stored in this database. “ Currently, analysts rely on 
their directors for distribution of sanction information, but they do not 
always receive such information, as we found in three of the ten cases we 
reviewed, described above. Officials who oversee the three schools also 
did not report checking any of the databases discussed above for this 
information. 

Information about sanctions may not be provided to analysts for review in 
some cases due to inconsistent information-sharing practices across 
Education’s eight regional divisions that provide oversight of schools that 
participate in federal student aid programs. Education officials we spoke 
with who oversee schools said that division directors are the key 
recipients of accreditor sanctions within the regional divisions, and that 
they should generally forward these sanctions to analysts responsible for 
overseeing the sanctioned schools. One division director we spoke with 
said she always did so, but an official from another division said her 
director would not necessarily forward a sanction if immediate action did 
not appear to be required. These inconsistencies in Education’s practices 
in providing accreditor sanctions to analysts for review may hinder its 
ability to monitor schools with accreditor sanctions. Moreover, even if it 
appeared that an action was not required by Education at the time a 
division director received a notification, this sanction information could still 
help inform the analyst overseeing the school about possible future 
actions. Federal internal control standards note that agencies should 
have relevant, reliable, and timely communications about internal and 
external events.®'* Because it is not clear whether Education 
communicated information about accreditor actions to the responsible 
officials, its analysts may have missed a chance to identify and respond 
to issues that could affect the administration of federal student aid funds. 

Furthermore, even when they review sanction data. Education staff who 
oversee schools do not always record decisions they make in response to 
an accreditor sanction in their working database, PEPS. Analysts who 


®®As noted earlier, the PEPS database contains Education’s oversight information of 
schools such as their eligibility for federal student aid programs and financial audit and 
student loan default rate data. Its purpose and users are completely separate from the 
accreditation database, mentioned above. The PEPS database contains limited 
information on accreditor sanctions that inform decisions by Education staff to take a 
specific monitoring or enforcement action against a school. 

®'*GAO/AIMD-00-21.3.1. 
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oversee schools that participate in federal student aid programs use 
PEPS to track the overall compliance of these schools, but reported that 
they do not always record their reviews of accreditor sanctions in the 
database, particularly when no action is necessary. However, Education 
officials acknowledged that the sanction review decisions — even the 
decisions for no action — can be significant and useful to the ongoing 
oversight of schools. According to federal internal control standards, 
information should be recorded and communicated to management and 
others who need it in a form and within a timeframe that enables them to 
carry out their responsibilities. By not recording its review of accreditor 
sanctions. Education limits its ability to systematically monitor schools 
with accreditation concerns. 

For 36 of the 93 schools receiving federal student aid funds that were 
placed on probation by their accreditors in fiscal year 2012, we found no 
indication of follow-up activities by Education between the beginning of 
fiscal year 2012 and December 2013.®®’®^ Not all accreditor sanctions 
require follow-up by Education, such as a sanction issued for failure to 
obtain student feedback. However, oversight actions by Education may 
be warranted if accreditor sanctions indicate potential federal student aid 
violations or other weaknesses affecting a school’s ability to appropriately 
administer federal student aid programs. As discussed above, our review 
of 10 schools with fiscal year 2012 accreditor sanctions found three cases 
in which analysts had no record of accreditor sanctions that could indicate 


®®GAO/AIMD-00-21.3.1. 

®®The oversight activities for which we requested data from Education included program 
reviews, heightened cash monitoring, letter of credit requirements, and termination of 
access to federal student aid. Program reviews are reviews of schools’ compliance with 
federal student aid standards. They may be initiated in response to potential performance 
problems or other risks exhibited by a school. Heightened cash monitoring may be 
imposed when schools have been determined not to be administering federal student aid 
appropriately. Schools on heightened cash monitoring are required to make federal 
student aid payments to students prior to: a) drawing down federal student aid funds, or b) 
submitting a request for reimbursement of those funds from Education. Schools that do 
not meet certain federal student aid requirements may also be required to post a letter of 
credit, which assures Education that funds would be available to make refunds to 
students, and pay obligations to the department and other expenses in the event that the 
school closed or ceased offering instruction. 

®^For 57 of the 93 sanctioned schools. Education records indicate that its analysts had 
undertaken some oversight action during this period. Education noted accreditation 
problems for 1 5 of the 93 schools that were receiving federal student aid funds and were 
placed on probation in fiscal year 2012. 
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a need for heightened federal student aid oversight. Because Education 
did not capture its decisions or the rationale for them in these cases, it is 
not possible to know if analysts did not review the cases at all, or if they 
reviewed them and determined that no action should be taken. 

Not consistently reviewing and recording decisions in response to 
accreditor sanction data also limits Education’s ability to systematically 
prioritize schools with accreditor sanctions for in-depth reviews (known as 
program reviews). The Higher Education Act requires that Education 
provide for the conduct of program reviews on a systematic basis, 
designed to include all schools participating in federal student aid 
programs. It provides that Education is to give priority to schools with 
certain specific risk factors, including those that have been reported to 
have deficiencies or financial aid problems by their accrediting agency.®® 
While Education selects schools for program review on both a case-by- 
case basis and as part of an annual selection process, the 
inconsistencies in reviewing and recording sanction information described 
above make it difficult for Education to ensure that it is selecting schools 
for program review based on complete information. For example, when 
selecting schools on a case-by-case basis. Education uses a risk matrix 
that includes accreditor sanctions to determine if a program review is 
appropriate. However, officials who oversee schools do not always 
receive sanction notices, and thus are not always aware when a school 
has been sanctioned. As a result, officials would not be able to 
systematically prioritize schools with sanctions for program review. 
Moreover, officials further noted that they have not considered including 
accreditor sanctions as a factor in their annual program review planning 
efforts. These efforts involve ranking and selecting schools for program 
review by indicators that Education has determined to reflect a likelihood 
of noncompliance with financial aid requirements. Education has 
incorporated other risk factors listed in law into this analysis but officials 
indicated that it has not used accreditor sanctions. These lapses in both 
types of selection processes make it difficult for Education to 


®®20 U.S.C. § 1 099C-1 (a)(1 ) and (2). The law also requires Education to prioritize schools 
for program review based on their default rates, changes in loan and grant volume, state 
licensing agency findings, dropout rates, and other factors that Education determines 
could put the school at significant risk of failing to comply with Title IV administrative 
capability or financial responsibility requirements. 
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Unclear Guidance 


systematically prioritize schools for review based on accreditor sanctions, 
as required by law.®® 

Unclear guidance from Education may also make it difficult for Education 
staff who oversee schools to respond consistently to these sanction 
notifications and contribute to lapses in oversight of schools, since the 
guidance does not lay out the recommended approach to specific types of 
accreditor sanctions.®® Moreover, although several officials who oversee 
schools told us they believed official guidance required them to restrict 
access to federal student aid funds for schools with show cause orders, 
the guidance does not specifically refer to show cause orders. In addition, 
the fact that Education may not have reviewed accreditor information 
about up to one-third of the 93 schools that were receiving federal student 
aid funds and that were placed on probation in fiscal year 2012, as 
discussed above, may also reflect the lack of clear guidance by sanction 
type. Having clear written guidance is consistent with federal internal 
control standards, which call for agencies to establish control activities 
that enforce management directives.®'' Officials we spoke with who 
oversee schools agreed that while they do have written guidance, their 
procedures are unclear, and noted that GAO’s study is prompting them to 
revise their guidance. 

Moreover, in part because Education’s guidance does not lay out the 
recommended approach to specific types of accreditor sanctions, officials 
who oversee schools do not consistently view accreditor sanction 
notifications as a valuable oversight tool. For example, one official noted 
that her team would never respond to accreditor probations because they 
occur too frequently to track and would disrupt other work.®^ However, our 
review found that just under 100 schools of the more than 6,000 
participating in federal student aid programs were placed on probation by 
their accreditor in fiscal year 2012. Another official said reviewing 
accreditor sanctions was not very useful in overseeing schools, as 


®®20 U.S.C. § 1099C-1 (a)(2)(D). 

®®Federal Student Aid, Financial Analysis Procedures (VJash'mgion, D.C.: Nov. 14, 2012), 
787. 

®^GAO/AIMD-00-21.3.1. 

®^This official also noted the importance of responding to show cause orders. However, 
these orders are not required to be reported to Education, and they vary in gravity. 
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accreditors would take additional action to prompt a response by 
Education if a school’s situation became more serious. However, other 
officials who oversee schools stated that they found show cause order 
notifications helpful.®^ Consequently, Education’s response to sanctions is 
inconsistent. Since accreditors may take other, informal steps prior to 
issuing a sanction, as discussed earlier in the report, accreditor sanctions 
can in fact be a serious indication of problems at a school. More 
specifically, all accreditor sanctions — including probations — can be an 
important source of information on schools. Consistent with federal 
internal control standards that call for ongoing, continual monitoring, 
reviewing accreditor sanctions in a timely manner can help analysts who 
oversee schools detect school compliance issues as they occur and 
prevent more serious problems from developing in the future.®'^ 


Education Does Not 
Systematically Use 
Sanction Data to Identify 
Accreditor Weaknesses 


Education does not systematically examine accreditor sanction data that 
could help identify insufficient accreditor oversight and thereby reduce 
potential risk to students and federal funds. Accreditors provide Education 
with records of terminations, probations, warnings, and show cause 
orders, which Education records in its database of accredited 
postsecondary schools.®® According to Education officials who oversee 
accreditors, this database was created in response to a GAO report on 
diploma mills in 2004 to help the public identify accredited schools.®® It 
contains information on the frequency and type of accreditor actions, 
which may provide insight during recognition reviews into an accreditor’s 


®®As noted earlier in the report, accreditors’ definitions of show cause may vary in 
seriousness. 

®4gAO/AIMD-00-21.3.1. 

®® Accreditors are required to notify Education of all terminations and probations that they 
issue, as well as denials of accreditation, suspensions, and withdrawals. 20 U.S.C. § 
1099b(a)(7). Accreditors voluntarily provide Education copies of some warning 
notifications and show cause orders. 

®®GAO, Diploma Mills: Diploma Mills Are Easily Created, and Some Have Issued Bogus 
Degrees to Government Employees at Federal Expense, GAO-04-1096T (Washington, 
D.C.: Sept. 23, 2004). Education uses the accreditor database that was created in 
response to this report to publish a list of accrediting agencies that the Secretary 
recognizes as reliable authorities on the quality of education or training provided by the 
schools they accredit. 
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ability to assess academic quality and appropriately identify schools for 
further oversight.®^ 

According to Education officials who oversee accreditors, they have not 
used sanction information for oversight because current regulations do 
not have specific criteria that require them to do so. While Education is 
not required to use sanction data or analyze accreditor sanctions as part 
of the recognition process, it could be useful for Education to consider 
these data when evaluating whether agencies meet prescribed criteria, 
such as whether accrediting agencies consistently apply and enforce 
standards.®® Federal internal control standards call for federal agencies to 
track data to help them make decisions, as well as conduct ongoing, 
consistent monitoring to identify weaknesses.®® The fact that Education 
does not broadly examine data for a given accreditor also makes it harder 
for the department to comprehensively identify potential risks and analyze 
them for possible effect, as required by federal internal control 
standards.^® In addition. Education’s own strategic plan calls for better 
use of data. While we are not suggesting using sanction data as a 
criterion for recognition, examining sanction information such as sanction 
rates for a given accreditor could provide another tool for Education to 
oversee the wide range of accreditors with varied and diverse member 
schools. Since accreditors are gatekeepers for tens of billions of dollars in 
federal student aid from Education as well as the key oversight bodies for 
ensuring academic quality, failure on the part of Education to spot 


®^Not all of this information is available to the public, which uses the database to obtain 
high-level information on whether a specific school is accredited. 

®®34C.F.R. §602.18. 

®®GAO/AIMD-00-21.3.1. 

According to these standards, risk identification methods may include, among other 
things, qualitative and quantitative activities and considering findings from assessments. 
Once risks have been identified, agencies should assess their significance and likelihood 
of occurrence. GAO/AIMD-00-21 .3.1 . 
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weaknesses in accreditor processes could result in poor quality schools 
gaining access to federal funds. 

More specifically, not systematically examining available sanction data 
deprives the department of a chance to detect problems that may not be 
apparent using the current recognition process. While these in-depth 
recognition reviews help Education assess accreditor oversight, they are 
limited because they rely mainly on evidence selected by accreditors 
themselves. For example. Education’s recognition guidelines allow 
accreditors to demonstrate that they appropriately issue sanctions by 
selecting and providing to Education examples of letters, meeting 
minutes, timelines, and required actions associated with specific 
sanctioned schools. Education officials who oversee accreditors stated 
that they recognize the limitations of allowing accreditors to submit the 
best examples they can to demonstrate compliance and noted it would 
not be difficult for an accreditor to hide problematic documentation. While 
a low sanction rate alone does not necessarily signal poor accreditor 
monitoring in all cases, it can serve as an important indicator of potential 
weak oversight on the part of accreditors, such as a failure to respond to 
schools that face academic quality or financial challenges. In fact. 
Education officials who oversee accreditors told us they value this 
information on accreditor sanctions, and explained that when they learn 
that an accreditor has issued few sanctions, they will take a closer look at 


^^Accreditation also affects eligibility for other federal higher education benefits provided 
by the Department of Veterans Affairs and the Department of Defense, since accreditation 
by an Education-recognized accreditor enables schools meeting other applicable 
requirements to participate in these programs. In fiscal year 201 3, the Department of 
Veterans Affairs provided over $12 billion in benefits for veterans’ postsecondary 
education. See GAO, VA Education Benefits: VA Shouid Strengthen its Efforts to Heip 
Veterans Make informed Education Choices, GAO-14-324 (Washington, D.C.: May 13, 
2014). In the same fiscal year, the Department of Defense provided $540 million through 
its military Tuition Assistance Program to help service members pursue a postsecondary 
education. See GAO DOD Education Benefits: Action is Needed to Ensure Evaiuations of 
Postsecondary Schoois Are Usefui, GAO-14-855 (Washington, D.C.: Sept. 8, 2014). 

^^In fiscal year 2015, Education will allow certain accreditors seeking re-recognition to 
submit shorter “focus reviews” to lessen the burden on accrediting agencies and to better 
focus Education’s assessment of these agencies’ effectiveness in ensuring academic 
quality. This streamlined process will enable these accreditors to demonstrate compliance 
with a smaller set of recognition criteria that Education considers most relevant to 
ensuring academic quality, including accreditor organization and administration, accreditor 
standards, application of standards, monitoring, enforcement, and how accreditors monitor 
substantive change at member schools. Education officials have not reported any plans to 
use sanction data to inform this process. 
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the accreditor and its oversight efforts. However, Education officials 
acknowledged that they have no systematic way of determining 
accreditors’ sanctioning rates, which can vary considerably. A more 
systematic examination of sanction data through the use of the database, 
with a follow-up that examines samples that are not selected by 
accreditors, could enhance this process. Indeed, our examination of 
accreditor sanctions from October 2009 through March 2014 showed that 
sanctioning rates among accreditors ranged from zero to 41 percent. In 
addition, one of the largest regional accreditors only issued one sanction 
in fiscal year 2012. In not utilizing these data. Education could be missing 
opportunities to determine if its recognized accreditors are effectively 
overseeing member schools that receive billions of dollars in federal 
funds each year. 

Education officials who oversee accreditors also expressed concern that 
analyzing data in the accreditation database would be too time- and 
labor-intensive. In particular, they state that to be of any value, an 
analysis of accreditor sanctions would have to examine the reasons 
behind the actions. According to Education officials who oversee 
accreditors, the department does not have the resources to undertake 
this effort. However, an analysis of these reasons need not be time- 
consuming, as the one that we undertook for all of fiscal year 2012 
sanctions involved just over 100 notices that were generally no longer 
than several paragraphs. Federal internal control standards state that the 
scope and frequency of monitoring can vary depending on the 
assessment of risk, effectiveness of ongoing monitoring, and rate of 
change within an entity and its environment.^^ Education could choose to 
further limit its focus by examining a sample of reasons for accreditor 
actions based on risks, such as accreditors with low sanctioning activities. 
Education officials reported that while they do not currently plan to 
conduct this analysis or any other potentially simpler ones, such as 
examining counts of accreditor sanctions, they are open to using 
additional available information to improve Education’s oversight of 
accreditors and place importance on developing an increasingly effective 
accreditation system. 


^^GAO/AIMD-OO-21.3.1. 


Page 39 


GAO-15-59 Higher Education Accreditation 


Conclusions 


While accreditation developed as a nongovernmental, peer review 
process, one of its main purposes now is to help ensure that the over 
$136 billion in federal student aid disbursed annually is invested wisely to 
help educate students at schools that provide a quality education. 
Although Education is prohibited from specifying the specific content of 
accreditor standards, the agency is responsible for assessing whether 
accreditors are effectively overseeing academic quality as part of their 
criteria for recognizing accreditors. However, our analysis found that 
accreditors were no more likely to sanction schools with weaker student 
outcomes than schools with stronger student outcomes. These findings 
raise questions about whether existing accreditor standards are sufficient 
to ensure the quality of schools, whether Education is effectively 
determining if these standards ensure educational quality, and whether 
federal student aid funds are appropriately safeguarded. 

While accreditor standards may be limited in enabling accreditors to 
effectively ensure academic quality. Education could nonetheless make 
better use of sanction information to determine whether schools are 
complying with federal financial aid requirements. Specifically, 
accreditors’ reviews of their member schools can provide valuable 
information to the government about instances when schools may have 
financial problems. Because Education does not consistently use 
accreditor sanction data in its oversight of schools, it may miss 
opportunities to identify and respond to school weaknesses that could put 
students and federal funds at risk. Clear guidance on how its analysts 
should consistently review, record, and respond to accreditor sanctions 
could enhance Education’s monitoring and oversight of schools. For 
example, ensuring that accreditor sanction information is easily 
accessible to analysts and that it is consistently reviewed, recorded, and 
responded to, where appropriate, could strengthen Education’s detection 
of potential federal student aid violations. Additionally, drawing upon 
accreditor sanction data to track sanctioned schools could help Education 
better focus its resources and prioritize schools with accreditor sanctions 
for program review on a systematic basis, as required by law. Since 
Education aims to conduct around 300 program reviews per year, it is 
particularly important that the department make judicious and well- 
informed choices about which schools to review out of the more than 
6,000 that participate in federal student aid programs. 

Similarly, without the use of data on accreditor sanctions. Education lacks 
information on trends and patterns that could indicate poor accreditor 
oversight of schools’ academic quality. Our review found that accreditor 
sanctioning rates can vary widely. Collecting and analyzing accreditor 
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data on the rate of sanction taking or doing additional analyses, such as 
using both sanction data and information collected by Education on 
student outcomes and financial characteristics, would help the 
department better determine if accreditors are consistently applying 
accreditor standards. 


Recommendations for ^ 
Executive Action 


To ensure that accreditors are reliable authorities on educational 
quality, we recommend that the Secretary of Education consider 
further evaluating existing accreditor standards to determine if they 
effectively address educational quality in key areas, such as student 
achievement. In carrying out this evaluation. Education could consider 
whether there are additional actions it could take, within the scope of 
its existing authority, to assess accreditor standards on an ongoing 
basis, and if appropriate. Education could develop a legislative 
proposal to expand its authority to assess accreditor standards. 


2. To strengthen Education’s oversight of schools, we recommend that 
the Secretary of Education ensure that Education staff consistently 
review, record, and respond to accreditor sanction information, and 
clarify its guidance on how the agency will respond to specific 
accreditor sanctions. 


3. To further strengthen Education’s oversight of schools, we 
recommend that the Secretary of Education determine how Education 
can consistently leverage available accreditor sanction data for 
oversight of schools to ensure it can systematically prioritize 
sanctioned schools for program review. 

4. To strengthen Education’s oversight of accreditors through the 
recognition review process, we recommend that the Secretary of 
Education draw upon accreditor data to determine whether 
accreditors are consistently applying and enforcing their standards to 
ensure that the education offered by schools is of sufficient quality. 

For example. Education could systematically use available information 
related to the frequency of accreditor sanctions or could do additional 
analyses, such as comparing accreditor sanction data with 
Education’s information on student outcomes, to inform its recognition 
reviews. 


Agency Comments 


Education provided us with written comments on a draft of this report, 
which are reproduced in appendix V. The agency agreed with three of our 
four recommendations, and partially concurred with one of them. 
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Education partially concurred with our recommendation that it consider 
further evaluating existing accreditor standards to determine if they 
effectively address educational quality, particularly in areas such as 
student achievement, and that it consider whether there are additional 
actions it could take, within the scope of its existing authority, to assess 
accreditor standards on an ongoing basis. We further recommended that 
if appropriate. Education develop a legislative proposal to expand its 
authority to assess accreditor standards. While Education stated that it 
will consider whether there are additional uses it can make of student 
achievement data in its accreditor oversight, it noted that the Higher 
Education Act and its implementing regulations currently do not supply 
any measure of the effectiveness of accreditor standards other than 
individual institutions meeting their stated objectives. Education stated 
that it intends to further evaluate existing accreditor standards as it 
formulates its recommendations for Higher Education Act reauthorization. 
We are pleased Education is considering whether it can make additional 
use of student achievement data in overseeing accreditors and that it will 
examine accreditor standards in the context of Higher Education Act 
reauthorization. In our report, we acknowledge that Education may face 
statutory limitations in strengthening accreditor standards, which is why 
we suggested that the department could consider developing a legislative 
proposal to expand its authority to assess accreditor standards. We hope 
that in its recommendations to Congress regarding Higher Education Act 
reauthorization. Education will include a request for any authority the 
department may need in order to fully assess the effectiveness of 
accrediting agency standards in evaluating educational quality. 

Education agreed with the remainder of our recommendations. The 
department noted that it is committed to identifying ways to use data 
about and from accreditors in its oversight, and it would provide a copy of 
our final report to the National Advisory Committee on Institutional Quality 
and Integrity. In addition to providing Education a draft copy of this report, 
we also provided relevant sections of the report to accreditors and 
associations we spoke with for review and comment. We received 
technical comments, which we incorporated as appropriate. 

We are sending copies of this report to the Secretary of Education, 
relevant congressional committees, and other interested parties. In 
addition, the report is available at no charge on the GAO website at 
http://www.gao.gov. 
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If you or your staff have any questions about this report, please contact 
me at (617) 788-0534 or emreyarrasm@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made key contributions to this 
report are listed in appendix VI. 

Sincerely Yours, 



Melissa Emrey-Arras, Director 

Education, Workforce, and Income Security Issues 
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Appendix I: Scope and Methodology 


Our research objectives were to examine: (1) what sanctions accreditors 
have taken against schools that did not comply with accreditor standards, 
(2) how likely accreditors are to sanction schools with weaker student 
outcomes or financial characteristics by terminating their accreditation or 
placing them on probation, and (3) how the Department of Education 
(Education) uses information on accreditor sanctions to support its 
oversight efforts. 


0v6rvi6W To determine the sanctions that accreditors took against postsecondary 

schools that did not comply with accreditor standards, we analyzed 
sanctions taken by accreditors against their schools from October 2009, 
when these data started to be consistently collected and reported, 
through when the latest data were released in March 2014, focusing on 
those reported to Education by accreditors.'' We included all probations 
and terminations (both of which are required to be reported to Education), 
as well as show cause orders (which indicate in some cases that a school 
must demonstrate why its accreditation should not be withdrawn) and 
warnings (which indicate a school does not meet one or more 
accreditation standards) that accreditors voluntarily reported to Education. 
We requested additional information on show cause orders and warning 
sanctions from accreditors that did not report these data to Education, 
and received responses from 27 of 31 accreditors. Based on these 
responses, we added an additional 90 show cause orders and 26 warning 
sanctions to our data set. However, we are not able to comment on 
whether this represents the full universe of show cause orders or warning 
sanctions. We used the Integrated Postsecondary Education Data 
System (IPEDS) to examine the characteristics — such as size and 
sector — of sanctioned schools and the students they serve. ^ 


^ Based on conversations with Education officials and a review of data from the 
accreditation database, we determined that the data in Education’s accreditation database 
were consistently collected and reported starting in fiscal year 2010. For this reason, our 
review of data covers from October 1, 2009 (the beginning of federal fiscal year 2010), 
through March 31 , 2014, the most recently available data at the time of our analyses. 

^Sector refers to whether a school is a public, nonprofit, or for-profit school, and size 
indicates whether a school offers programs of less than 2 years, 2 years, or 4 years or 
longer. 
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Appendix I: Scope and Methodology 


Reasons for Accreditor Sanctions 
Academic quality: issues with student 
achievement in relation to the mission and 
curricula, or other student outcomes. 
Administrative capability: issues such as 
those related to facilities, supplies, and 
administrative capability. 

Financial capability: issues with financial 
capability and compliance with federal 
student aid responsibilities. 

Integrity: fraud or misrepresentation. 
Governance: issues with division of 
responsibility, such as between the Board 
and a college president. 

Institutional Effectiveness: issues related 
to long-term plans for assessing learning and 
academic achievement. 

Other: other issues that do not fall in to the 
categories noted above. 

Source: GAO analysis of information from accreditors and 
Education. 


To develop our reasons categories we reviewed the accreditation 
standards of all seven regional accreditors and of two of the eight national 
accreditors.^ We also reviewed the statutory criteria that Education uses 
to recognize accrediting agencies. Based on this review, we developed 
seven categories for our reasons analysis: (1) academic quality; (2) 
financial responsibility; (3) administrative capability; (4) integrity; (5) 
governance; (6) institutional effectiveness; and (7) other (see side bar to 
left). 

In addition, we obtained feedback on our categories from Education 
officials and from eight of the accreditors whose standards we reviewed. 
Any one sanction could be taken for more than one reason. 

To develop a better understanding of the reasons behind sanctions, we 
analyzed all probation and termination notification letters sent to 
Education by accreditors for fiscal year 2012 and a nongeneralizable 
sample of other kinds of sanctions for that year.'^ For terminations, 
accreditors are required to provide Education with written notification 
related to the reasons for the sanctions. We also followed up with 
Education and accreditors to obtain notification letters for all probations in 
fiscal year 2012 and for our sample of other sanctions. In our analysis of 
these letters, we categorized the reasons for sanctions into seven 
categories, such as academic quality and financial capability.® 


To determine how likely accreditors are to sanction schools with weaker 
student outcome or financial characteristics, we conducted statistical 
modeling using data Education collected on schools to test whether 


®These two national accreditors were selected because they are two of the four largest 
national accreditors and reflect variation in the member schools they accredit. 

^We selected our sample from fiscal year 2012 because it was the most recent full year of 
data available when we started this analysis and obtained copies of all accreditor 
notification letters from Education. Because accreditors are not required to report show 
cause or warning actions to Education, we did not have a full universe of such actions. We 
selected a nonrandomized sample of 10 show cause and 10 warning sanctions from the 
125 show causes and 99 warnings reported by accreditors in Education’s database. We 
did this in order to include at least one sanction from each accreditor that reported show 
cause and/or warnings to Education. Any one sanction could be taken for more than one 
reason. This analysis was not generalizable. 

®For each notification letter, one analyst reviewed the letter and determined which reason 
categories it fit into, and then a second analyst reviewed and verified the categorization. 
Any discrepancies were discussed and resolved. 
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Appendix I: Scope and Methodology 


Selected Student Outcome 
Characteristics® 


schools with weaker characteristics are more likely to be terminated or 
placed on probation by accreditors. We analyzed characteristics data that 
Education collects in its IPEDS and other data systems, including the 
Postsecondary Education Participants System (PEPS), and that 
researchers have cited as indicators of academic quality, to evaluate 
whether accreditors were more likely to sanction schools that exhibited 
weaker characteristics in these two areas when compared to schools that 
did not.® We obtained input from Education on our list of characteristics. 
We examined accreditors’ overall responsiveness to student outcomes 
and financial characteristics — two key categories in which they are 
required to have standards — using statistical techniques that allowed us 
to compare schools’ findings concurrently across a range of related 
student outcome and financial characteristics, respectively. We also 
examined accreditors’ overall responsiveness to each individual student 
outcome and financial characteristic included in our analyses. The 
student outcomes and financial characteristics we used can be seen 
belowT 

• Three-Year Cohort Default Rate: the percent of borrowers in default 3 
years after entering repayment status. Education views this 
characteristic as an indicator of academic quality at schools, since 


^Accreditors set their own standards and are not required to assess school performance 
with respect to the information we used to examine the likelihood of sanctioning schools 
with results below any specific level in the areas we incorporated into our analysis. To 
ensure that we consistently compared how schools did on selected student outcome and 
financial characteristics, we used data that were collected by Education for all schools that 
participate in federal student aid programs. We used data from 2009 through 2014 since 
this was the most recent and complete accreditor sanction data available at the time of our 
review. We also considered including transfer rates but did not do so because they do not 
indicate whether a student has transferred to a 4-year program or to another 2-year 
program. Education does not have other national indicators of student success, such as 
graduate school applications and job placement or licensure exam pass rates. We limited 
our review to accreditors that accredit schools as opposed to unique programs within 
schools. 

^We defined weaker characteristics as being in the bottom one to five percent of the 
distribution for each individual characteristic, and stronger characteristics as being in the 
top one to five percent for each characteristic. We defined weaker characteristics as being 
in the bottom 25 percent of the distribution for the combined indicators for both the student 
outcome and the financial characteristics, and stronger ones as being in the top 25 
percent. 

®GAO analysis of key student outcome characteristics information collected by Education, 
based on program documents and interviews. 
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Selected Financial 
Characteristics® 


students who received a lower quality education may be less likely to 
have adequate income to repay their loans. 

Forbearance Rate: the percent of borrowers in forbearance (and 
therefore not repaying their loans on a temporary basis) during the 
official cohort default period. Education views this characteristic as an 
indicator of academic quality at schools, since students who received 
a lower quality education may be less likely to have adequate income 
to repay their loans. 

Graduation Rate: the percent of first-time full-time degree/certificate- 
seeking students who complete a program within 150 percent of the 
program length. A low graduation rate may indicate a lack of 
academic quality. 

Dropout Rate: the percent of students who left school during a 
particular year, but did not graduate. A high dropout rate may indicate 
a lack of academic quality. 

Retention Rate: the percent of first-time degree/certificate-seeking 
students who enrolled in one fall and either successfully completed 
their program or re-enrolled in the next fall. A low retention rate may 
indicate a lack of academic quality. 

Increases in Federal Student Aid: annual growth in federal student aid 
volume, which may indicate in extreme cases that growth may be too 
rapid to maintain academic and administrative services needed to 
adequately support students. 

Number of Program Review Findings: the number of findings at 
schools selected by Education for in-depth review due to the presence 
of certain risk factors, and the number of issues found in those 
reviews. 

Financial Health Scores: annual Education calculation of the financial 
health of for-profit and nonprofit schools using three ratios that 
capture different aspects of a school’s overall financial health. 

Program Reviews: schools selected by Education for in-depth review 
due to the presence of certain risk factors, many of which are 
financial, and the number of issues found in those reviews. 


®GAO analysis of key financial characteristics information collected by Education, based 
on program documents and interviews. 
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• Other Education Sanctions: schools sanctioned by Education for 
issues regarding the administration of federal student aid funds, many 
of which relate to financial performance and problems with a school’s 
financial stability. Sanctions may include heightened cash monitoring 
(requiring schools to disburse federal student aid funds before 
receiving funds) or a letter of credit (an assurance that financial 
obligations to Education would be met in the event of a school 
closure. 

• Decreases in Federai Student Aid: sharp declines in annual federal 
student aid volume may indicate financial instability for a school. 

• 90/10 Ratio: for-profit schools must report the percentage of their total 
revenues obtained from federal student aid funds (referred to as the 
90/10 rate). A high 90/10 ratio indicates high reliance on federal 
student aid funds and risk of non-compliance with a requirement that 
a for-profit school must obtain at least 10 percent of its revenue from 
non-Title IV sources. This could also indicate that a school is having 
difficulty obtaining other sources of funds and financial instability. 

For both types of analysis (combined indicators and each individual 
characteristic, respectively), we also separately examined how likely 
different types of accreditors — notably, regional versus national — were to 
sanction schools with weaker and stronger characteristics. We also 
examined whether accreditors responded to weaker characteristics 
differently at different types of schools (such as 2-year vs. 4-year public 
schools). For the student outcomes, we reviewed prior research on 
student outcomes, and because such research has shown that certain 
types of student body demographics may affect outcomes, when 
analyzing each unique outcome, we controlled for (1) students over 25 
years; (2) minority students; (3) students attending part time; and (4) low- 
income students — using the percentage of Pell grant recipients as a proxy 
for low income.'"’ (See page 50 for more about our methods.) 

To learn about how Education uses information on accreditor sanctions to 
support its oversight of schools and accreditors, we reviewed Education’s 
procedures for recognizing accreditors and overseeing schools, relevant 
federal laws and regulations, and interviewed Education officials. To 
obtain input regarding the accreditation process and Education’s 


^°See GAO, Postsecondary Education: Student Outcomes Vary at For-Profit, Nonprofit, 
and Pubiic Schoois, GAO-12-143 (Washington, D.C.: Dec. 7, 2011). 
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oversight, we also interviewed representatives from seven regional and 
two national accrediting agencies that have different oversight 
approaches, as well as the Council for Higher Education Accreditation, 
the Council of Regional Accrediting Commissions, and two schools with 
varying perspectives on the accreditation process. We also interviewed 
Education officials and representatives of the National Advisory 
Committee for Institutional Quality and Integrity, which advises the 
Secretary of Education on accreditation issues. We examined all 
schools that had been issued terminations and probations by their 
accreditor in fiscal year 2012, and drew upon data from PEPS to 
determine whether these sanctions prompted any follow-up monitoring or 
enforcement activities by Education, such as conducting program reviews 
or imposing restrictions on their access to federal student aid.''^ We also 
selected a nonrandomized sample of 10 schools that were sanctioned by 
their accreditors in fiscal year 2012, reviewed information about how 
Education monitored these schools and discussed with Education what 
action they took or did not take as the result of these sanctions. While 
this sample is not generalizable, we believe it provides valuable examples 
about how Education has responded to information from accreditors. 

To assess the reliability of the accreditation database, IPEDS, and PEPS 
data used in our analyses, we interviewed knowledgeable agency officials 
about the variables we used and conducted electronic testing to assess 
the reliability of these variables. We found these data sufficiently reliable 
for our purposes. We conducted this performance audit from July 2013 to 
December 2014 in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform the 
audit to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. We 
believe that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 


^^The Council for Higher Education Accreditation is an association of 3,000 colleges and 
universities that currently recognizes 60 accrediting agencies, ranging from large regional 
accreditors to small programmatic ones. The Council of Regional Accrediting 
Commissions is a consortium of regional accreditors. 

^^We selected schools with fiscal year 2012 accreditor sanctions because it was the most 
recent full year of information about the reason(s) for accreditor sanctions that was 
available when we began this analysis. 

^^We selected these sanctions in part because they indicated potential federal student aid 
concerns, and to capture a variety of Education response types. 
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Modeling the Accreditor 
Sanctioning Process 


We analyzed the association between various indicators of academic and 
financial risk (student outcomes and financial characteristics) and the 
likelihood that an accreditor sanctions a particular school participating in 
federal student aid programs. Through our interviews with accreditors, as 
well as our review of the literature and interviews with Education, we 
found that accreditors evaluate the quality of each school against its 
unique mission and the context in which it operates. Consequently, a 
single quantitative model of the decision to sanction a school — or even 
several types of models fit to different subsets of data — is unlikely to 
precisely reflect reality or fit the data closely. 

This type of comprehensive or causal modeling effort that we chose not to 
pursue has several limitations, given the many interactive and contextual 
factors involved in evaluating school quality. First, we would need to 
develop a strategy for accounting for all other variables that could 
obscure the relationship between the accreditor sanctions and each risk 
indicator, across multiple types of schools and accreditors. These 
confounding variables vary widely across schools and their accreditors, 
and also may vary depending on the indicator For example, a 2-year 
school with larger population of students who transfer to 4-year schools 
might have a lower graduation rate than a school with fewer students 
likely to transfer. This makes it difficult to credibly isolate the causal 
relationship between each indicator and accreditor sanctions. Second, the 
accreditation process considers various information about a school’s 
management and performance. Accreditors may consider quantitative 
data on school finances and student outcomes — the standardized data 
available from Education on schools nationwide — but accreditors also 
may consider more holistic or qualitative data on a school’s curriculum, 
faculty, administration, resources, and facilities. The latter can be difficult 
to examine and model quantitatively. 

Given the complexity of the accreditation process, we do not seek to 
comprehensively model the decision to sanction a school or to do a 
causal analysis. Rather, we seek to assess whether, in practice, schools 
with more risk on several plausible indicators are more likely to be 
sanctioned by their accreditors than schools with less risk. Accreditors 
may not literally consider a particular risk factor we analyze, such as 
graduation rates, when deciding whether to sanction a school. However, 
standardized indicators of quality, which are comparable and available 
across schools nationwide, are necessary for a quantitative assessment 
of how accreditors focus their sanctions. 
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We acknowledge that no single factor, or set of factors, necessarily 
should be correlated with sanctions across all schools receiving federal 
financial aid. Our approach balances the need to assess accreditors’ 
behavior against risk factors that Education views to be nationally 
important against a more nuanced evaluation process that considers the 
unique context of each school and information that cannot be precisely 
quantified. Our quantitative analysis is one part of our larger review, in 
which we consider the issues above in more detail. 

Variabl6S Usad As Risk We identified various indicators of risk by consulting with Education 

Indicators: Population of officials, by reviewing the literature on higher education accreditation and 

Interest and Data Sources academic quality, and by using professional judgment. Table 6 describes 

the risk indicators and provides basic statistics on their distribution in our 
data. The statistical analysis here does not seek to validate the risk 
indicators, but rather seeks to estimate their associations with the 
likelihood that an accreditor sanctions a particular school. 

The population of interest consists of postsecondary schools of higher 
education eligible to receive federal student aid funding from Education 
as of the 201 1-2012 school year. We identified this population by 
selecting the approximately 6,000 schools contained in IPEDS or in other 
Education databases that we used to obtain information on the risk 
indicators and that we could link to school accreditors. IPEDS is a system 
of interrelated surveys conducted annually by Education’s National 
Center for Education Statistics (NCES). These survey data contain 
information on school characteristics, enrollment, graduation rates and 
student financial aid. We assembled data on the risk indicators for each 
school from IPEDS and other Education sources. We combined the data 
using a common school identifier contained in all of the data sources. 

Complete data were not available for all schools and risk indicators, as 
shown in table 6. However, some of the risk indicators we identified did 
not necessarily apply to all schools in the population of interest. For 
example. Education only requires for-profit and nonprofit schools to 
calculate a financial composite score to examine financial strength, so no 
score is available for public schools. As another example, a school that 
formed in the last year would not have a current cohort default rate 
because the most current rate at the time of work was for loans issued 
during fiscal year 2010.We applied logic checks to the data to distinguish 
between missing values that were “not applicable” versus missing for 
other reasons. For example, graduation rates are based on students that 
complete their degree within 6 years. Therefore for schools that opened 
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after the 2005-2006 school year, the graduation rate would not be 
applicable. Similarly for analysis of the 90/1 0 ratios, which are only 
required of for-profit schools, these scores would be not applicable to 
public or not-for-profit schools. 

Rates of missing data for each indicator dropped to acceptable levels — 
approximately 0 to 13 percent — among schools for which the indicator 
applied, as shown in table 6. As a result, we concluded that excluding 
schools with missing data from the analysis would not substantially bias 
our findings. To preserve data for analysis, we conducted analyses using 
each risk factor individually as well as a multivariate analysis of all factors 
jointly, as described below. 


Table 6: Variables Used as Risk Indicators to Determine Likelihood of Accreditor Sanctions 


Risk Indicator 

Risk Scale 

% Schools 
with Missing 
Data 

% Schools 
with 

Indicator NA 

Mean 

Median 

Range 
Between 5th, 
95th Quantiles 

Skewness 

Kurtosis 

3-Year Loan 
Default Rate 
(2009) 

Academic 

3.1 

19.1 

13.7 

12.1 

[0.4,31.5] 

1.5 

9.2 

3-Year Loan 
Default Rate 
(2010) 

Academic 

4.5 

16.5 

14.6 

13.3 

[1.4,31.6] 

1.4 

8.8 

5-Year Loan 
Forbearance Rate 
(2009-2010 
Average) 

Academic 

0.1 

22.3 

43.6 

44.2 

[24.2,61.8] 

-0.2 

3.2 

Dropout Rate 
(2009) 

Academic 

13.4 

0.0 

54.9 

54.1 

[13.1,98.7] 

-0.0 

2.1 

Dropout Rate 
(2010) 

Academic 

10.9 

0.0 

55.4 

54.3 

[12.9,98.7] 

-0.0 

2.0 

Dropout Rate 
(2011) 

Academic 

8.9 

0.0 

54.4 

52.6 

[14,97.9] 

0.0 

2.0 

Dropout Rate 
(2012) 

Academic 

7.4 

0.0 

53.3 

51.3 

[13.1,96.8] 

0.1 

2.0 

Dropout Rate 
(2013) 

Academic 

6.3 

0.0 

51.6 

48.1 

[12.6,98.7] 

0.2 

2.1 

FSA Financial 
Composite Score 
(FY 2010) 

Financial 

10.7 

37.6 

2.4 

2.6 

[0.7,3] 

-1.7 

6.2 

FSA Financial 
Composite Score 
(FY 2011) 

Financial 

12.8 

37.3 

2.2 

2.4 

[0.8,3] 

-1.6 

6.5 

Graduation Rate 
(2011) 

Academic 

1.8 

16.1 

52.8 

54.7 

[12.5,91.1] 

-0.1 

2.0 
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Risk Indicator 

Risk Scale 

% Schools 
with Missing 
Data 

% Schools 
with 

Indicator NA 

Mean 

Median 

Range 
Between 5th, 
95th Quantiles 

Skewness 

Kurtosis 

Graduation Rate 
(2012) 

Academic 

2.1 

16.2 

52.6 

54.4 

[13,90.7] 

-0.1 

2.0 

Had 1+ FSA 
Sanction 

NA 

1.7 

0.0 

14.0 

0.0 

[0,100] 

2.1 

5.3 

Had 1+ Program 
Review Finding 

NA 

2.3 

0.0 

7.0 

0.0 

[0,100] 

3.4 

12.4 

Number of 
Program Review 
Findings (FY 2008- 
2010) 

Academic/Financi 

al 

0.7 

0.0 

1.3 

1.0 

[1.1] 

6.5 

46.9 

Number of 
Program Review 
Findings (FY 2011- 
2012) 

Academic/Financi 

al 

2.3 

0.0 

1.4 

1.0 

[1.4] 

4.9 

27.9 

On Heightened 
Cash Monitoring 
(FY 2008-2010) 

Financial 

0.0 

0.0 

6.8 

0.0 

[0,100] 

3.4 

12.7 

On Heightened 
Cash Monitoring 
(FY 2011-2012) 

Financial 

0.0 

0.0 

5.2 

0.0 

[0,100] 

4.0 

17.1 

Pot. Letters of 
Credit (2010-2012) 

Financial 

0.0 

0.0 

2.5 

0.0 

[0,25] 

4.6 

27.9 

Percent Decrease 
in Title IV Funding 
(FY 2010 to 2011) 

Financial 

1.4 

5.9 

-3.2 

0.0 

[-20,0] 

-4.8 

34.5 

Percent Decrease 
in Title IV Funding 
(FY2011 to 2012) 

Financial 

1.3 

3.8 

-7.7 

-0.4 

[-33.6,0] 

-2.8 

13.6 

Percent Increase in 
Title IV Funding 
(FY 2010 to 2011) 

Academic 

1.4 

0.0 

19.7 

8.0 

[0,63.9] 

9.1 

113.9 

Percent Increase in 
Title IV Funding 
(FY2011 to 2012) 

Academic 

1.3 

0.0 

11.4 

0.0 

[0,43.6] 

9.6 

121.9 

Retention Rate 
(2009) 

Academic 

8.0 

10.5 

70.3 

71.0 

[42.4,97.6] 

-0.5 

3.5 

Retention Rate 
(2010) 

Academic 

3.3 

16.2 

70.3 

71.1 

[43.1,95.6] 

-0.6 

3.9 


Source: GAO analysis of IPEDS data. | GAO-15-59 

Note: FY = fiscal year. 


Associations botwoon We use two statistical methods to assess the association between each 

Sanctions and for Spocific '"'Sk indicator and the likelihood of an accreditor sanction. 

Risk Indicators 
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For categorical risk factors, we use logistic generalized linear regression 
models to estimate the probability of at least one accreditor sanction, 
given each level of the indicator. Our models for examining academic 
quality (student outcomes) included controls for each school’s proportions 
of students who were Black, Hispanic, receiving Pell grants, over the age 
of 25, or attending part time. The use of controls produced better 
estimates of the association between each risk factor and the probability 
of a sanction, given variation in student characteristics across schools 
that may affect accreditors’ decisions. We parameterized each categorical 
risk factor as a series of indicator variables for each level of the variable, 
and parameterized the covariates as simple linear terms. Using the fitted 
model parameters, we calculated the in-sample average predicted 
probability of a sanction, fixing the risk factor to a particular level and 
averaging the predicted probabilities over the sample at the other 
covariates’ observed values. We used Monte Carlo simulation methods to 
estimate the uncertainty of this estimate. Specifically, we randomly drew 
1 ,000 sets of replicate parameters from the model’s fitted posterior 
distribution; estimated the in-sample average predicted probability for 
each replicate; and estimated 95 percent confidence intervals using the 
2.5 percent and 97.5 percent quantiles of simulated distribution. Our 
analyses for financial condition did not control for student demographics 
characteristics. 

For continuous indicators, we use a logistic generalized additive 
regression model. A logistic generalized additive regression model 
allowed for a non-parametric relationship between each continuous 
indicator and the probability of an accreditor sanction. Little previous 
research exists on the relationship between various risk indicators and 
accreditor sanctions, potentially because data on accreditors’ behavior 
are not publically available or analyzed by Education. As a result, we 
have little existing theory to specify a functional form ex ante. A 
nonparametric approach is consistent with the limited goal to describe 
patterns in the data, rather than to estimate a comprehensive statistical 
model or causal relationships. In addition, exploratory analysis showed 
that some indicators have unusual, non-linear relationships with the 
probability of accreditor sanctions. In particular, the probability of a 


14 

Trevor Hastie and Robert Tibshirani, 1986, “Generalized Additive Models,” Statistical 
Science 1 (3): 297-318 (1986); Simon Wood, 2011, “Fast, Stable Restricted Maximum 
Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear 
Models,” Journal of the Royal Statistical Society (B) 73 (1) (2011), 3-36. 
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sanction may increase sharply around the 80 to 90 percent value of 90/10 
scores, or the share of a for-profit school’s revenue from federal student 
aid programs. For-profit schools may not obtain 90 percent or more of 
their revenue from Title IV federal student aid programs. The yearly 
proportional change in federal student aid funding appeared to have a 
different relationship depending on whether the change was positive or 
negative and whether the change was large or small. 

Our logistic generalized additive regression model assumed that for 
school /, Pr( y, = 1 1 X) = exp(a + f(Z,) + xfi) / 1 + exp(a + f(Z,) + xfi), or 
equivalently that Logit( Y) | X) = a + f{Zi) + xfi. f{.) is an unspecified, 
smooth function of the risk factor Z„ rather than an additive linear function 
as in the typical logistic regression model. We assumed a simple linear 
model for the same covariates described above, denoted by xfi, which 
included an intercept. As with our analysis of categorical risk factors, we 
did not include covariates when the risk factor was financial. 

We estimated the model using standard penalized quasi-maximum 
likelihood methods and a cubic spline smoother for f(Z,).^® The 
penalization and smoothing parameter was estimated from the data using 
generalized cross-validation methods. The use of penalized likelihood 
and cross-validation methods helped avoid estimating f(Z,) in a manner 
that over-fit the sample or, equivalently, that chose an overly smooth 
functional form. We summarized the fitted model by estimating the 
predicted probability of a sanction at various quantiles of the risk factor of 
interest and calculated 95 percent confidence intervals using Monte Carlo 
simulation. Specifically, we drew 1,000 replicate parameters from of the 
fitted posterior distribution; calculated the in-sample average predicted 
probability at fixed values of the risk factor and averaging over the 
remaining covariates; and estimated confidence intervals using the 2.5‘^ 
and 97.5'^ quantiles of the simulated distribution of this quantity. These 
findings appear in tables 2-5 in the body of this report and figures 12-24 in 
appendices III and IV. 


^®For more details about these methods and our implementation of them, see Simon 
Wood, Generalized Additive Models: An Introduction with R. (Boca Raton, FL: Chapman 
and Hall/CRC Press, 2006). 
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Associations between 
Sanctions and Broad 
Indicators of Risk 


The factors we identified can be viewed as multiple ways to examine the 
risk of achieving potentially undesirable outcomes with federal student aid 
funding. Each variable we identified examines a particular type of risk, 
such as low retention rates or poor control over financial resources. 
Together, these outcomes may be combined to into broader indicators of 
risk that simultaneously reflect a school’s performance on many 
indicators. These broader indicators can more concisely summarize how 
accreditor sanctions relate to several aspects of risk at the same time. 

To conduct this broader analysis, we classified each risk factor into one of 
two dimensions of risk, based on discussions with Education and GAO 
staff subject matter expertise: “academic risk” and “financial risk.” The first 
and second columns of table 6 above describe how we categorized each 
indicator.''® 

We expressed the theoretical relationships between the risk factors and 
dimensions of risk as a Structural Equation Model, which has several 
benefits in this application. Structural Equation Model methods allowed us 
to empirically validate the existence of broad dimensions of risk and 
assess the reliability of the observed indicators as measures of those 
dimensions. Extending this “measurement model” to predict the 
probability of an accreditor sanction, in a “structural model,” jointly 
modeled the data available and propagated the measurement error and 
resulting uncertainty about the broader, or “latent,” indicators of risk. In 
addition. Structural Equation Model allowed for a multivariate analysis 
without using multiple intercorrelated controls, as in a generalized linear 
modeling approach that would assume that each indicator examines a 
unique dimension of risk without error. Finally, by acknowledging the 
imprecision of the risk indicators available for a large number of schools 
nationwide. Structural Equation Models avoided the need to rely on any 
particular indicator, such as graduation rates, that may be an imperfect 
indicator of student or administrative outcomes. 


^®We initially hypothesized that the risk factors measured three possible dimensions of 
risk: “academic risk,” “financial risk,” and “administrative risk.” After initial data analysis, we 
discovered that the factors we initially had classified as “administrative” or “financial” 
generally had similar intercorrelations. Moreover, we did not have a strong substantive 
reason to distinguish the concepts as distinct dimensions of risk. 
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Our model took the following form when expressed in the notation of a 
prominent textbook on Structural Equation Model 

Y = 7] 


X = + 5 





-All 0 - 




X = 

^12 

A = 

0 ^12 


II 

rH CN 

^12 


A,2- 


- 0 ^ri22. 



-^ri22. 


[Yi Yi] C = 


(il 

(2] 


The model consisted of a measurement model of how two latent variables 
of “academic risk” and “financial risk” (§) related to the observed risk 
indicators (x), as described by factor loading coefficients (A) and errors of 
measurement (5). The factor loading matrix was constrained to equal 
zero, except for the paths implied by the first and second columns of table 
6. Each indicator measured exactly one latent risk dimension, except for 
Education’s program review findings, which could reflect either 
dimension. (Program reviews can focus on either academic or financial 
issues). In addition, we hypothesized a structural model of how the latent 
risk variables related to the probability of an accreditor “probation” or 
“termination” sanction (indicator variable Y), as described by structural 
coefficients (f) and errors of prediction (^). Since accreditor sanctions is 
the variable of interest, we assumed that it was a perfect measure of a 
latent endogenous variable jy, with a measurement coefficient of 1 and a 
measurement error of 0. 

Table 6 above describes the variables used in the measurement model 
for each dimension of risk and provides descriptive statistics. Since most 
of the variables were scaled as percentages ranging potentially from 0 to 
100, we standardized all variables to vary on this range. We then scaled 
the Academic and Financial latent risk variables by fixing the 


^^Kenneth A. Bollen, Structural Equations with Latent Variables (New York, NY: John 
Wiley and Sons, 1989). 
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measurement coefficients of the default rate (2009) and financial health 
score (FY 2010) risk indicators to 1. The common scale of the indicators 
ensures that the unstandardized measurement coefficients are 
comparably scaled across indicators and can be roughly interpreted to 
range from 0 to 100. The model assumed that the means of the latent risk 
and error variables equaled 0 but estimated their variances as free 
parameters. 

Our data included risk indicators measured at several times, such as 
graduation rates for academic years 201 1 and 2012 (see table 6). Since 
these variables likely have similar measurement error across years, we 
allowed their measurement errors to be correlated across years. Apart 
from these variables, we assumed all measurement error to be 
independent. As a result, the measurement error covariance matrix of 5 
was block diagonal, with unique parameters expressing the covariance 
between the measurement error of each pair of repeatedly measured risk 
indicators and all other error covariances fixed to zero. 

We estimated the model separately for various subgroups of schools, 
according to their sector (public, for-profit, or nonprofit), their highest 
degree offered, and their use of regional versus national accreditors. 
Instead of estimating separate models (with different hypothesized latent 
variables or relationships) on subsets of the sample, we allowed the 
parameters of the model above to vary across the subgroups, which 
produced multiple parallel models. Our observed risk indicators may 
indicate the latent concepts of risk better for certain types of schools, 
such as four-year schools offering bachelor’s degrees. In addition, 
accreditors may be more or less likely to sanction a school with the same 
level of risk, depending on the nature of the school. Producing separate 
estimates across relevant subgroups allows for these potential 
substantive differences in the accreditation process. We omitted the 
financial risk dimension from models fit to public schools, since financial 
support from state and local governments suggests substantially different, 
and likely minimal, financial risk among public schools compared to 
private schools. 

We fit the model using robust quasi-maximum likelihood methods. This 
approach assumed that the endogenous observed variables, x, were 
jointly normally distributed, in order to estimate the factor loading 
coefficients. A, and structural coefficients, f. As table 6 suggests, one of 
our risk indicators — requirements for heightened cash monitoring — was 
discrete, and several were continuous but positively skewed, such as loan 
default rates. Even when applied to data that are not multivariate normally 
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distributed, quasi-maximum likelihood methods should produce consistent 
estimates of A and f,''® and robust covariance estimators relax the 
normality assumption when deriving the standard errors of these 
parameters.^® Alternative estimation methods, such as Asymptotic 
Distribution Free methods, proved difficult to converge, possibly due to 
the moderate size of our sample.^® 

Table 7 illustrates the fitted models for non-profit and for-profit schools, 
including parameter estimates and measures of fit. As described above, 
we fit similar models for other types of schools, but omit the generally 
similar findings to conserve space. The model fits the covariance matrix 
of the variables acceptably well, with a standardized root mean-squared 
residual of about 0.07 and coefficient of determination equal to 0.97. The 
measurement coefficients (factor loadings) are generally consistent with 
expectations and substantively large. The academic risk scale largely 
reflects dropout, graduation, and retention rates, with measurement 
coefficients of about 5 to 8 scale units (percentage points) in absolute 
value. Loan default and forbearance rates also load positively, but less 
strongly, on academic risk. The financial risk scale largely reflects 
Education’s financial composite score, but heightened cash monitoring, 
percent letters of credit, and, to a lesser extent, recent decreases in Title 
IV funding also load positively and moderately strongly. 

The model estimates that academic risk does not have a strong or 
statistically precise association with the probability of at least one 
accreditor sanction, but that financial risk has a significant and stronger 
association. A one-unit increase in the academic risk scale is associated 
with a 0.07 percentage point decrease in the probability of a sanction, but 
the association is not significantly distinguishable from zero. In contrast, 
financial risk is more strongly associated with the probability of a sanction. 
A one-unit increase in the financial risk scale is associated with a 0.55 
percentage point increase in the probability of at least one sanction, when 


1 ft 

Jonathan Nevitt and Gregory R. Hancock, “Performance of Bootstrapping Approaches to 
Model Test Statistics and Parameter Standard Error Estimation in Structural Equation 
Modeling,” Structural Equation Modeling 8 (3) (2001), 354. 

^®Andrew J. Tomarken and Niels G. Waller, “Structural Equation Modeling: Strengths, 
Limitations, and Misconceptions,” 1: 2.11 (2005). 

Tomarken and Waller, “Structural Equation Modeling”. 
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interpreting the regression of at least one sanction on the risk scales as a 
linear probability model. 


Table 7: Fitted Structural Equation Model of Academic Risk and the Probability of 
Accreditor Sanctions Against For-Profit and Nonprofit Schools 

Measurement Model 

Variable (x) 

Risk Scale (^) 

Estimate (A) 

Standard Error 

Number of Program Review 
Findings (FY 2008-2010) 

Academic 

.59 

.23 

Number of Program Review 
Findings (FY 2011-2012) 

Academic 

.26 

.19 

3-Year Loan Default Rate 
(2009) 

Academic 

1 (Fixed) 

Fixed 

3-Year Loan Default Rate 
(2010) 

Academic 

1.02 

0.09 

5-Year Loan Forbearance Rate 
(2009 - 2010 Average) 

Academic 

2.42 

0.27 

Dropout Rate (2009) 

Academic 

7.25 

1.00 

Dropout Rate (2010) 

Academic 

7.64 

1.06 

Dropout Rate (2011) 

Academic 

7.77 

1.08 

Dropout Rate (2012) 

Academic 

7.45 

1.02 

Dropout Rate (2013) 

Academic 

6.87 

.91 

Graduation Rate (2011) 

Academic 

-8.40 

1.19 

Graduation Rate (2012) 

Academic 

-8.61 

1.21 

Percent Increase in Title IV 
Funding (FY 2010 to 2011) 

Academic 

-.01 

.02 

Percent Increase in Title IV 
Funding (FY2011 to 2012) 

Academic 

.01 

.02 

Retention Rate (2009) 

Academic 

-4.78 

.63 

Retention Rate (2010) 

Academic 

-4.93 

.65 

FSA Financial Composite Score 
(FY 2010) 

Financial 

1 (Fixed) 

Fixed 

FSA Financial Composite Score 
(FY 2011) 

Financial 

.77 

.06 

Number of Program Review 
Findings (FY 2008-2010) 

Financial 

0.24 

0.07 

Number of Program Review 
Findings (FY 2011-2012) 

Financial 

.13 

0.04 

On Fleightened Cash 
Monitoring (FY 2008-2010) 

Financial 

.73 

.11 

On Fleightened Cash 
Monitoring (FY 2011-2012) 

Financial 

.77 

.11 
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Percent Letters of Credit (2010- 
2012) 

Financial 

.24 

.03 

Percent Decrease in Title IV 
Funding (FY 2010 to 2011) 

Financial 

.08 

.02 

Percent Decrease in Title IV 
Funding (FY2011 to 2012) 

Financial 

.12 

.03 

Structural Model 

Variable (Y) 

Risk Dimension (^) 

Estimate (F) 

Standard Error 

Flaving 1+ Sanction 

Academic 

-.07 

.26 

Flaving 1+ Sanction 

Financial 

.55 

.11 

Model Fit Statistics 

Standardized Root Mean-Squared Residual: .073 

Coefficient of Determination: .97 


Source: GAO analysis of IPEDS and other Education databases. | GAO-15-59 


Notes: The table illustrates the fitted model for-profit and nonprofit schools only. See page 50 for a 
complete description of the models estimated, including those fit for different types of schools (e.g., 
public schools). The table entries are a subset of fitted parameters, excluding intercepts, error 
variances, and error covariances. FY = fiscal year. 

In tables 2 through 5, above, we illustrate the results of this model in an 
accessible way by estimating the academic and financial/administrative 
risk factor scores for each school in the sample, grouping these scores 
into quartiles, and then estimating the proportion of schools receiving at 
least one sanction. 
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Figure 10: Sanctions against Schoois (percent) by Schooi Type and Program Length, October 2009 through March 2014 


4-year or above 1 | 2-year 


<1 

A 

<1 

2 

A 

2 

A 

o 

For-profit 

Public 

lir 

Nonprofit 

For- 

profit 

• 

Public 

w 

Nonprofit 

Probation 

Public 

o 

Nonprofit 

2 

• 

For- 

profit 

o 

Public 

2 

• 

Nonprofit 

o 

For-profit 


Less than 2-year 




Show Cause 


• 

Nonprofit For- 

profit 



Warning 


2 


Nonprofit 


1 

For- 

profit 




1 1 

-• 

Public Nonprofit 


For- 

profit 


Source; GAO analysis of data from the Integrated Postsecondary Education Data System (IPEDS) and the Department of Education’s accreditation database. | GAO-1 5-59 


Note: Percentages may not add to 100 because of rounding. Program length refers to the longest 
program a school has, but it may also have shorter programs. The show cause orders and warnings 
depicted by this graph were voluntarily reported to Education or obtained from accreditors by GAO. 
Unlike terminations and probations, show cause orders and warnings are not required to be reported 
to Education. 
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Figure 11: School-level Student Characteristics by Type of Sanction, October 2009 through March 2014 
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Likelihood of Accreditor 
Sanctions for Weaker 
Versus Stronger Student 
Outcomes 


Figure 12: Accreditor Likeiihood of Termination or Probation for Schoois with Weaker vs. Stronger Student Outcomes, by 
Type of Accreditor, October 2009 through March 2014 
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Source: GAO analysis of school-level student outcome characteristics collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to concurrently examine accreditors’ likelihood 
of sanctioning schools with weaker versus stronger characteristics across one combined indicator 
incorporating a range of related student outcome characteristics (schools in the top 25 percent vs. 
bottom 25 percent). It was not possible to divide for-profit and nonprofit schools by type of accreditor 
or public schools accredited by national accreditors due to small sample size. In addition, we also 
analyzed how likely programmatic accreditors and other accreditors — including state agencies that 
act as accreditors — were to sanction member schools based on these outcomes. Because these 
accreditors had relatively few member schools, we were unable to determine the statistical 
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significance of these results for just programmatic or other small accreditors alone, but they are 
included in our overall results. See appendix I for more details. 

^Indicates that the difference between estimates at the top and bottom 25 percentiles is statistically 
distinguishable from 0 at the 95 percent confidence level with the exception of the difference for 
regional accreditors of nonprofit and for-profit schools, which is statistically distinguishable from 0 at 
the 90 percent confidence level. 


Figure 13: Accreditor Likelihood of Termination or Probation, by 3-Year Default Rate, October 2009 through March 2014 


Overall 

Percentage likelihood of sanction 
10 



1% 5% 10% 25% 50% 75% 90% 95% 99% 

Lowest Middle Highest 

Default rate (in percentiles) 


Regional 


8 . 3 ® 



1% 5% 10% 25% 50% 75% 90% 95% 99% 

Lowest Middie Highest 

Default rate (in percentiles) 


National 


9.9 



1% 5% 10% 25% 50% 75% 90% 95% 99% 

Loivesf Middie Highest 

Default rate (in percentiles) 


Source; GAO analysis of school-level student outcome characteristics collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker individual student outcome characteristics, compared to schools with 
stronger characteristics. Schools with weaker student outcomes were considered to be those in the 
bottom vs. the top for each characteristic (those in the 1st vs. 99th percentile and 5th vs. 95th 
percentile). We controlled for the following demographic characteristics: percentage of part-time 
students, percentage of students over the age of 25 year, percentage of minority students, and 
percentage of students receiving Pell grant aid. Default rate indicates the percent of borrowers who 
entered repayment in fiscal years 2009 or 201 0 and were in default as of the end of the second 
following fiscal year. In addition, we also analyzed how likely programmatic accreditors and other 
accreditors — including state agencies that act as accreditors — were to sanction member schools 
based on their outcomes. Because these accreditors had relatively few member schools, we were 
unable to determine the statistical significance of these results for just programmatic or other small 
accreditors alone, but they are included in our overall results. See appendix I for more details. 

^Indicates that the difference between estimates at the 1st and 99th percentiles and the 5th and 95th 
percentiles, respectively, is statistically distinguishable from 0 at the 95 percent confidence level. 
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Figure 14: Accreditor Likelihood of Termination or Probation, by Graduation Rate, October 2009 through March 2014 
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Source; GAO analysis of school-level student outcome characteristics collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker individual student outcome characteristics, compared to schools with 
stronger characteristics. Schools with weaker student outcomes were considered to be those in the 
bottom vs. the top for each characteristic (those in the 1st vs. 99th percentile and 5th vs. 95th 
percentile). We controlled for the following demographic characteristics: percentage of part-time 
students, percentage of students over the age of 25 year, percentage of minority students, and 
percentage of students receiving Pell grant aid. Graduation rates reported to IPEDS in 201 1 and 2012 
are for first-time full-time degree/certificate-seeking undergraduate students that completed their 
degree within 150 percent of the expected time. In addition, we also analyzed how likely 
programmatic accreditors and other accreditors — including state agencies that act as accreditors — 
were to sanction member schools based on their outcomes. Because these accreditors had relatively 
few member schools, we were unable to determine the statistical significance of these results for just 
programmatic or other small accreditors alone, but they are included in our overall results. See 
appendix I for more details. 

^Indicates that the difference between estimates at the 1st and 99th percentiles and the 5th and 95th 
percentiles, respectively, is statistically distinguishable from 0 at the 95 percent confidence level. 
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Figure 15: Accreditor Likelihood of Terminations or Probations, by Dropout Rate, October 2009 through March 2014 
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Source: GAO analysis of school-level student outcome characteristics collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors' likelihood of 
sanctioning schools with weaker individual student outcome characteristics, compared to schools with 
stronger characteristics. Schools with weaker student outcomes were considered to be those in the 
bottom vs. the top for each characteristic (those in the 1st vs. 99th percentile and 5th vs. 95th 
percentile).We controlled for the following demographic characteristics: percentage of part-time 
students, percentage of students over the age of 25 year, percentage of minority students, and 
percentage of students receiving Pell grant aid. Dropout rate indicates the total number of 
withdrawals reported by each school during a particular year divided by the total number of graduates 
plus withdrawals reported to the National Student Loan Data System for that year for award years 
2008-2009 through 2012-2013. In addition, we also analyzed how likely programmatic accreditors 
and other accreditors — including state agencies that act as accreditors — were to sanction member 
schools based on their outcomes. Because these accreditors had relatively few member schools, we 
were unable to determine the statistical significance of these results for just programmatic or other 
small accreditors alone, but they are included in our overall results. See appendix I for more details. 

^Indicates that the difference between estimates at the 1st and 99th percentiles and the 5th and 95th 
percentiles, respectively, is statistically distinguishable from 0 at the 95 percent confidence level. 
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Figure 16: Accreditor Likelihood of Termination or Probation, by Retention Rate, October 2009 through March of 2014 


Overall 

Percentage likelihood of sanction 
15 

12 



1% 5% 10% 25% 50% 75% 90% 95% 99% 

Lowest Middle Highest 

Retention rate (in percentiles) 


Regional 


8.4= 



1% 5% 10% 25% 50% 75% 90% 95% 99% 

Lowest Middle Highest 

Retention rate (in percentiles) 


National 


14.2 



1% 5% 10% 25% 50% 75% 90% 95% 99% 

Lowest Middle Highest 

Retention rate (in percentiles) 


Source: GAO analysis of school-level student outcome characteristics collected by Education and data from the accreditation database. | GAO-1 5-59 


Notes: We used statistical techniques that allowed us to examine accreditors' likelihood of 
sanctioning schools with weaker individual student outcome characteristics, compared to schools with 
stronger characteristics. Schools with weaker student outcomes were considered to be those in the 
bottom vs. the top for each characteristic (those in the 1st vs. 99th percentile and 5th vs. 95th 
percentile). We controlled for the following demographic characteristics: percentage of part-time 
students, percentage of students over the age of 25 year, percentage of minority students, and 
percentage of students receiving Pell grant aid. Retention rate indicates the percent of first-time 
degree/certificate-seeking students who enrolled in the previous fall and either successfully 
completed their program or re-enrolled in the next fall as reported to IPEDS in the fall of 2010 and 
201 1 . In addition, we also analyzed how likely programmatic accreditors and other accreditors — 
including state agencies that act as accreditors — were to sanction member schools based on their 
characteristics. Because these accreditors had relatively few member schools, we were unable to 
determine the statistical significance of these results for just programmatic or other small accreditors 
alone, but they are Included in our overall results. See appendix I for more details. 

^Indicates that the difference between estimates at the 1st and 99th percentiles and the 5th and 95th 
percentiles, respectively, is statistically distinguishable from 0 at the 95 percent confidence level. 
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Figure 17: Accreditor Likelihood of Termination or Probation, by Average Forbearance Rate, October 2009 through March 
2014 
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Source; GAO analysis of school-level student outcome characteristics collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker individual student outcome characteristics, compared to schools with 
stronger characteristics. Schools with weaker student outcomes were considered to be those in the 
bottom vs. the top for each characteristic (those in the 1st vs. 99th percentile and 5th vs. 95th 
percentile). We controlled for the following demographic characteristics: percentage of part-time 
students, percentage of students over the age of 25 year, percentage of minority students, and 
percentage of students receiving Pell grant aid. Forbearance rate indicates the percent of borrowers 
who entered repayment status in fiscal year 2009 and 2010 and were in forbearance as of the end of 
the following fiscal year. In addition, we also analyzed how likely programmatic accreditors and other 
accreditors — including state agencies that act as accreditors — were to sanction member schools 
based on their characteristics. Because these accreditors had relatively few member schools, we 
were unable to determine the statistical significance of these results for just programmatic or other 
small accreditors alone, but they are included in our overall results. See appendix I for more details. 
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Figure 18: Accreditor Likeiihood of Termination or Probation against For-Profit and Nonprofit Schoois with Weaker vs. 
Stronger Financiai Characteristics, by Type of Accreditor, October 2009 through March 2014 


Overall for-profit 



stronger 

(Top 25%) 


0.0 

Overall nonprofit 



Stronger 

(Top 25%) 


Regional 


Weaker 

(Bottom 25%) 

Stronger 

(Top 25%) 


Weaker 

(Bottom 25%) 

Stronger 

(Top 25%) 


0 . 0 ^ 

National 


6 . 1 ® 


0 . 0 ® 

0 5 

Percentage likelihood of sanction 


25.4® 


20 . 8 ® 


10 


15 


20 


25 


Source: GAO analysis of financial characteristics information collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us concurrently examine accreditors’ likelihood of 
sanctioning schools with weaker versus stronger characteristics across one combined indicator 
incorporating a range of related financial characteristics (top 25 percent vs. bottom 25 percent). It was 
not possible to divide for-profit and nonprofit schools by type of accreditor due to small sample size 
and public schools are not required to have composite scores since they are generally financed by 
state and local entities. In addition, we also analyzed how likely programmatic accreditors and other 
accreditors — including state agencies that act as accreditors — were to sanction member schools 
based on their financial characteristics. Because these accreditors had relatively few member 
schools, we were unable to determine the statistical significance of these results for just 
programmatic or other small accreditors alone, but they are included in our overall results. We did not 
examine the combined financial characteristics of public schools because Education does not 
measure their financial health, as they rely, in large part, on state and local funding. See appendix I 
for more details. 

^Indicates that the difference between estimates at the top and bottom 25 percentiles is statistically 
distinguishable from 0 at the 95 percent confidence level. 
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Figure 19: Accreditor Likelihood of Termination or Probation, by Average Financial Health Score, October 2009 through 
March 2014 
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Source; GAO analysis of financial characteristics information collected by Education and data from the accreditation database. | GAO-1 5-59 


Notes: We used statistical techniques that allowed us to examine accreditors' likelihood of 
sanctioning schools with weaker individual financial characteristic, compared to schools with stronger 
characteristics. Schools with weaker financial characteristics were considered to be those in the 
bottom vs. the top for each characteristic (those in the 1st vs. 99th percentile and 5th vs. 95th 
percentile). Financial health score refers to a school’s fiscal year 201 0 and 201 1 financial composite 
score, which is calculated by Education using three key financial ratios examining different aspects of 
a school’s financial health. In addition, we also analyzed how likely programmatic accreditors and 
other accreditors — including state agencies that act as accreditors — ^were to sanction member 
schools based on their characteristics. Because these accreditors had relatively few member schools, 
we were unable to determine the statistical significance of these results for just programmatic or other 
small accreditors alone, but they are included in our overall results. We did not examine this financial 
characteristic for public schools because Education does not measure their financial health, as they 
rely, in large part, on state and local funding. See appendix I for more details. 

’’Indicates that the difference between estimates at the 1st and 99th percentiles and the 5th and 95th 
percentiles, respectively, is statistically distinguishable from 0 at the 95 percent confidence level. 
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Figure 20: Accreditor Likelihood of Termination or Probation, Based on Program Review by Education, October 2009 through 
March 2014 
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Source: GAO analysis of financial characteristics information collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker individual financial characteristic, compared to schools with stronger 
characteristics. This variable is a dichotomous yes/no variable, and indicates whether a school had a 
program review with at least one finding in fiscal years 201 1 or 2012. Many findings are financially 
related. In addition, we also analyzed how likely programmatic accreditors and other accreditors — 
including state agencies that act as accreditors — were to sanction member schools based on their 
characteristics Because these accreditors had relatively few member schools, we were unable to 
determine the statistical significance of these results for just programmatic or other small accreditors 
alone, but they are included in our overall results. See appendix I for more details. 

^Indicates that the difference between estimates is statistically distinguishable from 0 at the 95 
percent confidence level. 


Page 72 


GAO-15-59 Higher Education Accreditation 


Appendix IV: Data on the Likelihood of 
Termination or Probation for Selected Federal 
Financial Characteristics 


Figure 21 : Accreditor Likelihood of Termination or Probation, by Number of Program Review Findings, October 2009 through 
March 2014 
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Source: GAO analysis of financial characteristics information collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker individual financial characteristics compared to schools with stronger 
characteristics. This variable is a dichotomous yes/no variable, to indicate the number of findings from 
any program reviews between October 1 , 2010 and September 30, 2012. In addition, we also 
analyzed how likely programmatic accreditors and other accreditors — including state agencies that 
act as accreditors — were to sanction member schools based on their characteristics. Because these 
accreditors had relatively few member schools, we were unable to determine the statistical 
significance of these results for just programmatic or other small accreditors alone, but they are 
included in our overall results. See appendix I for more details. 

^Indicates that the differences between estimates are statistically distinguishable from 0 at the 95 
percent confidence level for schools with 0 findings, 1-9 findings, or 10 or more findings, respectively. 
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Figure 22: Accreditor Likelihood of Termination or Probation, by Sanctions Taken by Education Related to Financial 
Concerns, October 2009 through March 2014 
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Source: GAO analysis of financial characteristics information collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors' likelihood of 
sanctioning schools with weaker individual financial characteristics compared to schools with stronger 
characteristics.. This is a dichotomous yes/ no variable and examines if a school was on heightened 
cash monitoring (requiring schools to disburse federal student aid funds before receiving funds), was 
required to have a letter of credit (an assurance that financial obligations to Education would be met 
in the event of a school closure), or had a program review with at least one finding in fiscal years 
201 1 or 2012. In addition, we also analyzed how likely programmatic accreditors and other 
accreditors — including state agencies that act as accreditors — were to sanction member schools 
based on their characteristics. Because these accreditors had relatively few member schools, we 
were unable to determine the statistical significance of these results for just programmatic or other 
small accreditors alone, but they are included in our overall results. See appendix I for more details. 

^Indicates that the difference between estimates is statistically distinguishable from 0 at the 95 
percent confidence level. 
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Figure 23: Accreditor Likelihood of Termination or Probation, by Changes in Federal Student Aid Funds, October 2009 
through March 2014 
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Source: GAO analysis of financial characteristics information collected by Education and data from the accreditation database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker individual financial characteristics compared to schools with stronger 
characteristics. For this characteristic, weaker characteristics were considered to be in the bottom vs. 
the median or the top vs. the median (those in the 1st or 5th vs. 50th percentile and 95th and 99th vs. 
50th percentile. Change in federal student aid funding indicates a decrease in the amount of funding 
a school received. Change in Title IV funding indicates any decrease in federal student aid funds. In 
addition, we also analyzed how likely programmatic accreditors and other accreditors — including state 
agencies that act as accreditors — ^were to sanction member schools based on their characteristics. 
Because these accreditors had relatively few member schools, we were unable to determine the 
statistical significance of these results for just programmatic or other small accreditors alone, but they 
are included in our overall results. See appendix I for more details. 

^Indicates that the estimate is statistically distinguishable from the estimate at the 50th percentile at 
the 95 percent confidence level. 
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Figure 24: Accreditor Likelihood of Termination or Probation, by 90/10 Ratio, 
October 2009 through March 2014 
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Source: GAO analysis of financial characteristics information collected by Education and data from the accreditation 
database. | GAO-15-59 


Notes: We used statistical techniques that allowed us to examine accreditors’ likelihood of 
sanctioning schools with weaker individual financial characteristics compared to schools with stronger 
characteristics. For this characteristic, weaker characteristics were considered to be in the bottom vs. 
the median or the top vs. the median (those in the 1st or 5th vs. 50th percentile and 95th and 99th vs. 
50th percentile. We looked at if a for-profit school met the requirement to receive not more than 90 
percent of its revenues from federal student aid funds for fiscal years 2010 and 201 1 . This 
characteristic is not applicable to nonprofit or public schools. In addition, we also analyzed how likely 
programmatic accreditors and other accreditors — including state agencies that act as accreditors — 
were to sanction member schools based on their characteristics. Because these accreditors had 
relatively few member schools, we were unable to determine the statistical significance of these 
results for just programmatic or other small accreditors alone, but they are included in our overall 
results. See appendix I for more details. 
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UNITED STATES DEPARTMENT OF EDUCATION 
OFFICE OF POSTSECONDARY EDUCATION 

THE ASSISTANT SECRETARY 

December 10, 2014 


Ms. Melissa Emrey-Arras 
Director, Education, Workforce, 
and Income Security Issues 
United States Government Accountability Office 
Washington, DC 20548 

Dear Ms. Emrey-Arras: 

Thank you for providing the Department of Education (Department) with a draft copy of the U.S. 
Government Accountability Office’s (GAO’s) report entitled “HIGHER EDUCATION: 
Education Should Strengthen Oversight of Schools and Accreditors” (GAO-15-59; Job Code 
131270). The report examines aspects of the effectiveness of accreditation in the Title IV 
programs — a topic of intense interest and ongoing consideration within the Department. The 
Department is giving in-depth attention to this area as Congress considers the reauthorization of 
the Higher Education Act (HEA). We also share GAO’s interest in maximizing the use of 
available data to inform decision making, and are committed to identifying ways to use data 
about and from accreditors in our oversight of postsecondary institutions and agencies. As part 
of our efforts, we intend to provide a copy of GAO’s final report to the National Advisory 
Committee on Institutional Quality and Integrity, an advisory committee of experts appointed by 
the Department and both houses of Congress, for comment and as a catalyst for 
recommendations in these areas. 

GAO focused the present study on accreditor sanction data. GAO found that accreditor 
sanctions did not coirelate with student outcome data GAO selected, but that accreditors were 
more likely to sanction institutions with weaker financial characteristics. GAO also found 
disparity among recognized accreditors in terms of how often they terminated accreditation or 
placed institutions on probation. From these findings, GAO made four recommendations 
regarding the Department’s use of information on accreditor sanctions to support oversight of 
institutions and agencies. It is important to note that the regulatory authority of the Office of 
Postsecondary Education (OPE) differs from that of Federal Student Aid (FSA). OPE regulates 
accrediting agencies while FSA’s authority applies to educational programs offered at 
institutions. 

Recommendation: To ensure that accreditors are reliable authorities regarding educational 
quality, we recommend that the Secretary of Education consider farther evaluating existing 
accreditor standards to determine if they effectively address educational quality in key areas, 
such as student achievement. In carrying out this evaluation, Education could consider whether 
there are additional actions it could take, within the scope of its existing authority, to assess 


1990 K ST. N.W., WASHINGTON, DC 20006 
www.ed.gov 

The Department of Education’s mission is to promote student achievement and preparation for global competitiveness 
by fostering educational excellence and ensuring equal access. 
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accreditor standards on an ongoing basis, and if appropriate. Education could develop a 
legislative proposal to expand its authority to assess accreditor standards. 

Response: The Department concurs in part with this recommendation. The Department intends 
to further evaluate existing accreditor standards as it formulates its recommendations regarding 
HEA reauthorization. In addition, the Department will consider whether there are additional 
uses it can make of student achievement data in its procedures for review of accrediting agencies, 
understanding that the HEA and its implementing regulations currently do not supply any 
measure of the effectiveness of accreditor standards other than individual institutions meeting 
their stated objectives. The HEA and regulations also prevent the Department from prescribing 
the content of agency standards. 

Recommendation: To strengthen Education ’s oversight of schools, we recommend that the 
Secretary of Education ensure that Education staff consistently review, record, and respond to 
accreditor sanction information, and clarify the guidance it provides staff on responding to 
accreditor sanctions. 

Response: The Department concurs with this recommendation. While FSA does use accreditor 
sanction information in its oversight, to help ensure the consistent review, recording, and 
response to accreditor sanction iirformation, FSA will strengthen the current Eligibility National 
Standards and provide additional staff training. 

Recommendation: To further strengthen Education ‘s oversight of schools, we recommend that 
the Secretary of Education determine how Education can consistently leverage available 
accreditor sanction data for oversight of schools to ensure it can systematically prioritize 
sanctioned schools for program review. 

Response: The Department concurs with this recommendation. FSA will incorporate accreditor 
sanctions into the annual risk assessments used to select program review candidates, and tvill 
strengthen the Program Review National Standards as appropriate. 

Recommendation: To strengthen Education 's oversight of accreditors through the recognition 
review process, we recommend that the Secretary of Education draw upon accreditor data to 
determine whether accreditors are consistently applying and enforcing their standards to ensure 
that the education offered by schools is of sufficient quality. For example. Education could 
systematically use available information related to the frequency of accreditor sanctions or 
could do additional analyses, such as comparing accreditor sanction data with Education 's 
information on student outcomes, to inform its recognition reviews. 

Response: The Department concurs with this recommendation. The Department will 
systematically track the number of accreditor sanctions issued by each accrediting agency against 
the institutions they accredit. Those data will then be used to allow us to focus our limited 
resources on those accrediting agencies with extremely low or high sanction rates. 
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The Department appreciates the attention GAO is giving the critical issue of accreditation and 
shares the interest in ensuring that institutions participating in Federal programs are of high 
academic quality. You wiU find enclosed some technical comments based on our review of this 
report. 


Sincerely, 

Lynn B. MahafFie 
Acting Assistant Secretary 


Enclosure 
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